slide 1 ecmwf data assimilation training course – may 2010 ensemble data assimilation massimo...

Slide 1

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Ensemble Data Assimilation

Massimo Bonavita

ECMWF

Acknowledgments: Lars Isaksen, Elias Holm, Mike Fisher,

Laure Raynaud

Slide 2

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Outline

•The Ensemble Data Assimilation method

•What do we do with the EDA?

•Use of EDA variances in ECMWF 4DVar

•4DVar assimilation experiments

•Conclusions and plans

Slide 2

Slide 3

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Outline

•The Ensemble Data Assimilation method

•What do we do with the EDA?

•Use of EDA variances in 4DVar

•4DVar assimilation experiments

•Conclusions and plans

Slide 3

Slide 4

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

“When simulating the error evolution of the reference system one should use the reference gain matrix K” (Berre et al. 2007)

The EDA method

KyxKHIx fa

ofa KeeKHIe

Slide 4

Slide 5

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

This is what an ensemble of 4DVar analyses with random observation and SST perturbations does!

εa/f/o = perturbations w.r.to ensemble mean

The EDA method

ofa KεεKHIε

Analysis

xb+εb

y+εo

SST+εSST (etc.)

xa+εa

Forecastxf+εf

Slide 5

Slide 6

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• After a few cycles the EDA system will have forgotten the initial background perturbations

• We are diagnosing the background error statistics of the actual analysis system

However:

• We assume that observation and SST errors are correctly specified

• We need to account for the model error component of the background error

The EDA method

Slide 6

Slide 7

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

“Stochastic Kinetic Energy Backscatter”

SKEB (Berner et al., 2009)

ΔXperturbed physics= f(KE,ΔXphysics,n)

A fraction of the dissipated energy is backscattered upscale and acts as stream function forcing on resolved-scale flow.

Spectral Markov chain: temporal and spatial correlations prescribed

Only vorticity perturbed, so only wind field directly affected

All levels are perturbed. Either randomly or partially correlated.

“Stochastically Perturbed Parameterisation Tendencies”

SPPT (Leutbecher, 2009)

ΔXperturbed physics= ( 1+μr) ΔXphysics

Random pattern r varies smoothly in space and time, with de-correlation scales 500 km and 6 hours.

Gaussian distribution with no bias and stdev 0.5 (limited to ±3stdev)Same random number r

for X=T, q, u, v

No perturbations in lowest 300 m and above 50 hPa (0≤ μ ≤1).

from: Lars Isaksen

The EDA method

Slide 7

Slide 8

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

The EDA methodComparing the impact of various stochastic perturbation methods: zonal mean

u wind ensemble background forecast spread

N S

Slide 8

Impact of SKEB on U spread Additional Impact of SPPT on U spread

from: Lars Isaksen

Slide 9

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

The EDA methodComparing the impact of various stochastic perturbation methods: zonal mean

Temperature ensemble background forecast spread

N S

Slide 9

Impact of SKEB on T spread Additional Impact of SPPT on T spread

from: Lars Isaksen

Slide 10

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

•In the EDA context model error parametrizations should increase spread where/when errors are larger => should increase spread-error correlation

•Are they doing this?

Slide 10

The EDA method

Slide 11

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

EDA (baseline) - EDA+SKEBEDA+SPPT - EDA+SPPT+SKEB

Slide 11

The EDA method

Slide 12

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

EDA (baseline) - EDA+SKEBEDA+SPPT - EDA+SPPT+SKEB

Slide 12

The EDA method

Slide 13

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

•Model error parametrizations do increase spread-error correlations

•Large effect in the lower troposphere Tropics

•Small effect in the Extratropics

•Similar behaviour-performance of the two schemes and their combination

Slide 13

The EDA method

Slide 14

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

10 ensemble members using 4D-Var assimilations

T399 outer loop, T95/T159 inner loop (reduced number of iterations)

Observations randomly perturbed

Cloud track wind (AMV) correlations taken into account

SST perturbed with realistically scaled structures

Model error represented by stochastic methods (SPPT, Leutbecher, 2009)

All 107 conventional and satellite observations used

The EDA method

from: Lars Isaksen

Slide 14

Slide 15

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Outline

•The Ensemble Data Assimilation method

•What do we do with the EDA?

•Use of EDA variances in ECMWF 4DVar

•4DVar assimilation experiments

•Conclusions and plans

Slide 15

Slide 16

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• The EDA simulates the error evolution of the 4DVar analysis cycle. As such it can be applied to:

1. Compute climatology of B matrix for use at the initial time of 4DVar (Analysis-Ensemble method, Fisher 2003; see talk on B matrix modelling)

2. Provide initial conditions for ensemble forecasts (EPS)

3. Provide a flow-dependent sample of background errors at the initial time of 4DVar

What do we do with the EDA?

Slide 16

Slide 17

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Improving Ensemble Prediction System by including EDA perturbations for initial uncertainty

The Ensemble Prediction System (EPS) benefits from using EDA based perturbations. Replacing evolved singular vector perturbations by EDA based perturbations improve EPS spread, especially in the tropics.The Ensemble Mean has slightly lower error when EDA is used.

What do we do with the EDA?

Slide 17

EVO-SVINIEDA-SVINI

EVO-SVINIEDA-SVINI

N.-Hem. Tropics

Ensemble spread and Ensemble mean RMSE for 850hPa T

Slide 18

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• The EDA simulates the error evolution of the 4DVar analysis cycle. As such it can be applied to:

1. Compute climatology of B matrix for use at the initial time of 4DVar (Analysis-Ensemble method, see talk on B matrix modelling)

2. Provide initial conditions for ensemble forecasts (EPS)

3. Provide a flow-dependent sample of background errors at the initial time of 4DVar

What do we do with the EDA?

Slide 18

Slide 19

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• 4DVAr does not cycle error information (B), only the state estimate

• 4DVar behaves like a Kalman filter in which the covariance matrix is reset to some static matrix B every few hours (12h in ECMWF implement.)

• Two possible ways around this:

1. Run 4DVar over a long enough window so that the influence of the initial state and errors on the final state analysis is negligible

Slide 19

What do we do with the EDA?

Slide 20

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

T+120S.hem Lat -90.0 to -20.0 Lon -180.0 to 180.0

Root mean square error forecast500hPa Geopotential

Time series curves

15

AUGUST 200516 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

30

40

50

60

70

80

90

100

110

120

all obs

all obs

Analysis experiment started with satellite data reintroduced on 15th August 2005

Analysis experiment started with satellite data reintroduced on 15th August 2005

From Mike Fisher

Memory of the initial statedisappears after approx. 3 days

Slide 20

Slide 21

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• An analysis window of ≥3 days would allow 4DVar analysis at final time to be independent of initial state and error estimates

• However:

1. An effective model error representation must be applied to reconcile model and measurements over a long analysis time window (i.e., weak constraint 4DVar)

2. We would still lack an estimate of analysis errors

Slide 21

What do we do with the EDA?

Slide 22

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• Two possible ways around this:

1. Run 4DVar over a long enough window so that the influence of the initial state and errors on the final state analysis is negligible

2. Use a sequential method (EDA, EnKF) to cycle error covariance information

Slide 22

What do we do with the EDA?

Slide 23

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Hybrid methods

• Use of ensemble perturbations in a 3-4DVar analysis

• Cycle error information through ensemble of Data Assimilations

• Retain the implicit full rank error representation of 3-4DVar

µεσον τε και αριστονAristotle, Nic. Ethics 2.6

Slide 23

Slide 24

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Hybrid systems

• Ensemble perturbations can be used in a 3-4DVar analysis in a number of different ways:

1. Use ensemble variances for observation QC

2. Use ensemble (co)variances as starting B matrix of minimization (often in linear combination with climatological B, extra control variable)

3. Use of ensemble covariances inside 4DVar minimization (En4DVAR)

Slide 24

What do we do with the EDA?

Slide 25

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Outline

•The Ensemble Data Assimilation method

•What do we do with the EDA?

•Use of EDA variances in ECMWF 4DVar

•4DVar assimilation experiments

•Conclusions and plans

Slide 25

Slide 26

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• We want to use EDA perturbations to simulate 4DVar flow-dependent error covariance evolution

• We start with the diagonal of the Pf matrix, i.e.:

“Estimate the first guess error variances with the variance of the EDA short range forecasts”

• This has been tried before (Kucukkaraca and Fisher, 2006, Fisher 2007, Isaksen et al., 2007) but results have been inconclusive

Use of EDA variances in 4DVar

Slide 26

Slide 27

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Define: Veda = EDA sampled variance

V* = True error variance

Then the estimation error can be decomposed:

Veda-V* = E[Veda-V*] + (Veda-E[Veda])

systematic random

Similarly for mean square of the estimation error:

E[(Veda-V*)2] = (E[Veda-V*])2 + E[(Veda-E[Veda])2]

systematic random

We should try to minimize both!Slide 27

Use of EDA variances in 4DVar

Slide 28

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

What raw ensemble variances look like?

Spread of Vorticity FG t+9h ml=64

Slide 28

Use of EDA variances in 4DVar

Slide 29

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• Noise level is due to sampling errors: 10 member ensemble

• EDA is a stochastic system: variance of variance estimator ~ 1/Nens

• We need a system to effectively filter out noise from first guess ensemble forecast variances: Reduce the random component of the estimation error

Slide 29

Use of EDA variances in 4DVar

Slide 30

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

“Mallat et al.: 1998, Annals of Statistics, 26,1-47”

Define Ge(i) as the random component of the sampling

error in the estimated ensemble variance at gridpoint i:

Then the covariance of the sampling noise can be shown to be a simple function of the expectation of the ensemble-based covariance

matrix:

(1)

iiiie BEBiG

~~

2~

1

2ij

ee BEN

jGiGE

Slide 30

Use of EDA variances in 4DVar

Slide 31

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

“Mallat et al.: 1998, Annals of Statistics, 26,1-47”

A consequence of (1) is that:

(2)

i.e., sampling noise is smaller scale than background

error. If the variance field varies on larger scales then the background error

=> we can use a spectral filter to disentangle noise

error from the sampled variance field

2

iLiL b

ee

G

Slide 31

Use of EDA variances in 4DVar

Slide 32

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010 Slide 32

Use of EDA variances in 4DVar

Slide 33

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• There is indeed a scale separation between signal and sampling noise!

• Truncation wavenumber is determined by maximizing signal-to-noise ratio of filtered variances (details in Raynaud et al., 2009, and forthcoming Tech. Memo)

• Optimal truncation wavenumber depends on parameter and model level

Slide 33

Use of EDA variances in 4DVar

Slide 34

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Raw Ensemble StDevVO ml64

Filtered Ensemble StDevVO ml64

Slide 34

Use of EDA variances in 4DVar

Slide 35

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Raw Ensemble StDevSpec. Hum. ml64

Filtered Ensemble StDevSpec. Hum. ml64

Slide 35

Use of EDA variances in 4DVar

Slide 36

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Operational StDevRandom. Method (Fisher & Courtier, 1995

Filtered Ensemble StDevVO ml64

Slide 36

Use of EDA variances in 4DVar

Slide 37

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Is Filtering the Ensemble Variances enough to improve the analysis?

Not quite…

N.Hem. Z 500hPA AC S.Hem. Z 500hPA AC

Slide 37

Use of EDA variances in 4DVar

Slide 38

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Should we also do something about the systematic component of the estimation

error?

E[(Veda-V*)2] = (E[Veda-V*])2 + E[(Veda-[Veda])2]

systematic random

A statistically reliable ensemble satisfies:

Slide 38

Use of EDA variances in 4DVar

Mean_Errord_Ens_Square_VarianceEns11

1111

ensens NN

Slide 39

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Vorticity ml 30 (~50hPa) Ensemble Error Ensemble Spread

Spread - Error

Slide 39

Use of EDA variances in 4DVar

Slide 40

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Vorticity ml 78 (~850hPa) Ensemble Error Ensemble Spread

Spread - Error

Slide 40

Use of EDA variances in 4DVar

Slide 41

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Is the ensemble fg statistically calibrated?

• Calibration factors needs to be model level, latitude and parameter dependent

• Calibration factors seems also to be flow-dependent, i.e. depend on the size of the expected error

Slide 41

Use of EDA variances in 4DVar

Slide 42

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010 Slide 42

Use of EDA variances in 4DVar

Slide 43

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• Calibration factors need to be flow-dependent, too!

• Do they also change in time?

Slide 43

Use of EDA variances in 4DVar

Slide 44

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010 Slide 44

Use of EDA variances in 4DVar

Slide 45

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010 Slide 45

Use of EDA variances in 4DVar

Slide 46

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• There is not a large day-to-day variability but seasonal variability is important

• General solution: slowly varying adaptive calibration coefficients

Slide 46

Use of EDA variances in 4DVar

Slide 47

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Variance post-process

xa+εia

Analysis ForecastSST+εi

SST

y+εio

xb+εib

xf+εif

i=1,2,…,10

EDA Cycle

εif raw

variances

Variance Recalibration

Variance Filtering

EDA scaled variances

4DVar Cycle

xa

Analysis ForecastEDA scaled Varxb xf

Slide 47

Slide 48

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010 Slide 48

Slide 49

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Outline

•The Ensemble Data Assimilation method

•What do we do with the EDA?

•Use of EDA variances in ECMWF 4DVar

•4DVar assimilation experiments

•Conclusions and plans

Slide 49

Slide 50

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

4DVar assimilation with EDA variances

Deterministic DA experiments with EDA variances

1. CY35r3_esuite, T799L91, 7/01 – 16/02 2009

2. Control f8a4

3. Experiment fb4k with ensemble DA variances:

a) Calibration step: adaptive, flow-dependent, regionally varying, for each parameter and model level

b) Filtering step: “Optimal” spectral filtering

c) EDA variances are used both in observation QC and start of 4DVar minimization

Slide 50

Slide 51

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

4DVar assimilation with EDA variances

Slide 51

Slide 52

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

4DVar assimilation with EDA variances

N.HEM Z acJan-Feb

Slide 52

Slide 53

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

4DVar assimilation with EDA variances

S.HEM Z acJan-Feb

Slide 53

Slide 54

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

4DVar assimilation with EDA variances

TROP. VW RMSEJan-Feb

Slide 54

Slide 55

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

4DVar assimilation with EDA variances

Deterministic DA experiments with EDA variances

1. CY36r2_esuite, T1279L91, 7/08 – 16/09 2008

2. Control far9

3. Experiment fb9x with ensemble DA variances:

a) Calibration step: adaptive, flow-dependent, regionally varying, for each parameter and model level

b) Filtering step: “Optimal” spectral filtering

c) EDA variances are used both in observation QC and start of 4DVar minimization

Slide 55

Slide 56

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

4DVar assimilation with EDA variances

N.HEM Z acAug-Sep

Slide 56

Slide 57

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

4DVar assimilation with EDA variances

S.HEM Z acAug-Sep

Slide 57

Slide 58

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

4DVar assimilation with EDA variances

TROP VW RMSEAug-Sep

Slide 58

Slide 59

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

4DVar assimilation with EDA variances

Deterministic DA experiments with EDA variances

1. Results are generally positive in the Extra-Tropics (more so in the NH)

2. Small positive impact in the Tropics

Slide 59

Slide 60

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

4DVar assimilation with EDA variances

Slide 60

• Observation departures statistics support the idea that (slightly) different observation usage is the result of smarter OBS QC decisions

Slide 61

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Outline

•The Ensemble Data Assimilation method

•What do we do with the EDA?

•Use of EDA variances in ECMWF 4DVar

•4DVar assimilation experiments

•Conclusions and plans

Slide 61

Slide 62

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

• First step towards an error cycling DA

• Use of flow dependent EDA variances has the potential to improve the deterministic scores

• A careful post-processing step of the raw ensemble first guess forecast is necessary to:

a)Filter sampling noise

b)Adaptively calibrate the ensemble

Conclusions and plans

Slide 62

Slide 63

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

•Better OBS QC decisions seems to be playing a part

•Further improvements in model error parameterizations will directly benefit the system

•Increase in ensemble size will benefit the system

Conclusions and plans

Slide 63

Slide 64

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

WHERE NEXT

•Operational implementation and testing

•Further tuning of system at full operational resolution (T1279L91)

Conclusions and plans

Slide 64

Slide 65

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

•Generalize the use of EDA variances to unbalanced components of control vector

•Relax the assumption: analysis=“truth” in the calibration step

Conclusions and plans

Slide 65

Slide 66

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Medium term:

•Refine the representation of initial uncertainties (correlated perturbations, surface fields uncertainties) in stochastic EDA

•Evaluate EnKF covariances

•Further develop the hybridization of 4DVar with EDA (investigate the use of EDA covariances)

Conclusions and plans

Slide 66

Slide 67

ECMWF Data Assimilation Training Course – May 2010 ECMWF Data Assimilation Training Course – May 2010

Thanks for your attention!

I welcome your questions/comments…

Slide 67