a localized markov chain particle filter (lmcpf) for the ... · a localized markov chain particle...

A Localized Markov Chain Particle Filter (LMCPF)

for the Global Weather Prediction Model ICON

Anne Sophie Walter and Roland Potthast and Andreas Rhodin

German Meteorological Service (DWD)

Data Assimilation Unit

7th International Symposium on Data Assimilation 2019

[email protected] Localized Markov Chain Particle Filter for the Global Weather Prediction Model ICON 2

What is new?

• Two localized particle filters working stably in an operational setup: LAPF and

LMCPF

• Strong improvements of scores by including model error via a Gaussian

mixture approach for the prior distribution

• LMCPF partly decouples spread and assimilation functionality (via model error)

• First particle filter tests with full global operational resolution and setup,

including two-way European nest in global DA

• Particle Filter shows behavior similar to LETKF, with some scores better than

LETKF, some worse (o-b and forecast!)

• LMCPF also works similarly for COSMO convective scale

22.01.2019


Content

1. The ICON-Model and DA-System

2. The Localized Markov Chain Particle Filter (LMCPF)

3. Numerical Testing

4. Summary

22.01.2019


1. The ICON-Model

• Operational NWP model of DWD: ICON (ICOsahedral Nonhydrostatic)

• Two-way NEST over Europe (~6.5 km)

• Resolution: 13 km deterministic

40 km ensemble (40 member)

22.01.2019


1. DA System at DWD

• Hybrid EnVar operational since

January 20, 2016

𝑋𝑎 = ҧ𝑥𝑏 + 𝑋𝑏𝑊

• Following Hunt et al. (2007),

(see also Schraff et al., 2016)

• EnVar-B-Matrix: 70% LETKF,

30% Climatology

22.01.2019

B-Matrix


2. The LMCPF

• Our Particle Filters are based on the LETKF philosophy

➢ Localization: as LETKF in ensemble space

➢ Adaptivity: tools for spread control

22.01.2019

PRIOR

DATA

Posterior

Analysis

Ensemble

➢ Able to handle with multi-modal

distributions

➢ Able to handle with Non-Gaussianity


2. The LMCPF

22.01.2019

LETKF

Classical PF

DATA

Prior

Posterior

DATA

Prior

Posterior

LMCPF

DATA

Prior

Posterior


2. The LMCPF

• Employ Bayes formula to calculate new analysis distribution

𝑝𝑘𝑎

𝑥 := 𝑝 𝑥 𝑦𝑘 = 𝑐 𝑝 𝑦𝑘 𝑥 𝑝𝑘𝑏

𝑥 , 𝑥 ∈ ℝ𝑛

𝑐 is a normalization factor: 𝑋 𝑝𝑘𝑎

𝑥 𝑑𝑥 = 1

• To carry out the analysis step at time 𝑡𝑘 a posteriori weights 𝑤𝑘,𝑙𝑎

are

calculated

𝑤𝑘,𝑙(𝑎)

= 𝑐 𝑒−12 𝑦−𝐻𝑥 𝑙 𝑇

𝑅−1(𝑦−𝐻𝑥 𝑙 )

𝑐 is chosen such that σ𝑙=1𝐿 𝑤𝑘,𝑙

(𝑎)= 𝐿

22.01.2019

First Step: The Classical Particle Filter


2. The LMCPF

• Accumulated weights 𝑤𝑎𝑐 are defined:

𝑤𝑎𝑐0 = 0

𝑤𝑎𝑐𝑙 = 𝑤𝑎𝑐𝑙−1 +𝑤𝑙𝑎 , 𝑙 = 1,… , 𝐿

where 𝐿 denotes the ensemble size

• Drawing 𝑟𝑙~𝑈 0,1 , 𝑙 = 1,… , 𝐿, set 𝑅𝑙 = 𝑙 − 1 + 𝑟𝑙 and define transform

matrix ෲ𝑾 for the particles by:

ෲ𝑊𝑖,𝑙 = ൝1 𝑖𝑓 𝑅𝑙 ∈ 𝑤𝑎𝑐𝑙−1 , 𝑤𝑎𝑐𝑙 ,

0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒,

𝑖, 𝑙 = 1,… , 𝐿 with 𝑊 ∈ ℝ𝐿𝑥𝐿, (𝑠, 𝑡] denotes the interval of values 𝑠 < 𝜂 ≤ 𝑡.

22.01.2019

Second Step: Classical Resampling


2. The LMCPF

• Based on the adaptive multiplicative inflation factor 𝝆 determined by the

LETKF

𝜌 =Ε 𝒅𝑜−𝑏

𝑇 𝒅𝑜−𝑏 − Tr(𝐑)

𝑇𝑟(𝑯𝑷𝑏𝑯𝑇)

• Weighting factor 𝜶 has been chosen, due to the small ensemble size (𝐿 = 40)

𝜌𝑘 = 𝛼 𝜌𝑘 + 1 − 𝛼 𝜌𝑘−1

22.01.2019

Third Step: Spread Control


2. The LMCPF

• Perturbation factor 𝜎 is used to add spread to the system

𝜎 =

𝑐0, 𝜌 < 𝜌(0)

𝑐0 + 𝑐1 − 𝑐0 ∗𝜌 − 𝜌(0)

𝜌(1) − 𝜌(0), 𝜌(0) ≤ 𝜌 ≤ 𝜌(1)

𝑐1, 𝜌 > 𝜌(1)

where 𝑐0 = 0.02, 𝑐1 = 1.5, 𝜌(0) = 1.0 and 𝜌(1) = 1.4, with 𝜎 = 𝑐1if 𝜌 ≥ 𝜌(1) and

𝜎 = 𝑐0 if 𝜌 ≤ 𝜌(0)

22.01.2019

Third Step: Spread Control


2. The LMCPF

෨𝐵 = 𝐼 − 𝐾𝐻 𝐵

= 𝐼 − 𝐵𝐻𝑇 𝑅 + 𝐻𝐵𝐻𝑇 −1𝐻 𝐵

= 𝐼 − 𝛾𝑋 𝑋𝑇𝐻𝑇 𝑅 + 𝛾 𝐻𝑋 𝑋𝑇𝐻𝑇 −1𝐻 𝛾𝑋𝑋𝑇

= 𝑋(𝐼 − 𝛾 𝑌𝑇(𝑅 + 𝛾𝑌𝑌𝑇)−1𝑌)𝛾𝑋𝑇

= 𝑋 𝐼 − 𝛾 𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 −1𝑌𝑇𝑅−1𝑌 𝛾X𝑇

= 𝑋 𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 −1(𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 − 𝛾𝑌𝑇𝑅−1𝑌) 𝛾𝑋𝑇

= 𝑋 𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 −1𝛾𝑋𝑇

= 𝛾𝑋 𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 −1𝑋𝑇

22.01.2019

Fourth Step: Determine posterior B ≡ ෨𝐵

𝐾 = 𝐵𝐻𝑇 𝑅 + 𝐻𝐵𝐻𝑇 −1

𝐵 = 𝛾𝑋𝑋𝑇

𝐻𝑋 = 𝑌

𝑌𝑇 𝑅 + 𝛾𝑌𝑌𝑇 −1 = 𝐼 + 𝛾𝑌𝑇𝑅−1𝑌 −1𝑌𝑇𝑅−1


2. The LMCPF

𝛽(𝑠ℎ𝑖𝑓𝑡,𝑙) = 𝐼 + 𝛾Y𝑇𝑅−1𝑌 −1𝑌𝑇𝑅−1 𝐶 − 𝑒𝑙

with 𝐴 = 𝑌𝑇𝑅−1𝑌, 𝐶 = 𝐴−1𝑌𝑇𝑅−1(𝑦0 − ത𝑦𝑏)

𝑊(𝑠ℎ𝑖𝑓𝑡) ≔ 𝛽 𝑠ℎ𝑖𝑓𝑡,1 , … , 𝛽(𝑠ℎ𝑖𝑓𝑡,𝐿) ∈ ℝ𝐿×𝐿

Important effects of 𝛽(𝑠ℎ𝑖𝑓𝑡,𝑙):

• it moves the particles towards the observations in ensemble space

• by the use of model error, it constitutes a further degree of freedom which can be used for

tuning of a real system

22.01.2019

Fifth Step: Calculation of the Shift-Vector 𝛽(𝑠ℎ𝑖𝑓𝑡,𝑙)


2. The LMCPF

• Weights W are calculated by drawing from the posterior.

𝑊 = 𝑊 +𝑊(𝑠ℎ𝑖𝑓𝑡) ∗ 𝑊 + ෨𝐵 ∗ 𝑅𝑛𝑑 ∗ 𝜎

22.01.2019

Sixth Step: Gaussian Resampling

with 𝑅𝑛𝑑 normally distributed random numbers,

𝑊(𝑠ℎ𝑖𝑓𝑡) and ෨𝐵 calculated with Gaussian radial

basis function (rbf) Approximation for prior

density and observation error

DATA

Prior

Posterior

➢ It is an explicit calculation of the Bayes posterior based on radial basis

function approximation of the prior.

Centers of posterior RBFs Shape of posterior RBFs


• Full ensemble: 40 members

• Reduced resolution:

• 26km deterministic

• 52km ensemble

• Period:

06.05.2016 – 31.05.2016

• Full ensemble: 40 members

• Routine resolution:

• 13km deterministic

• 40km ensemble

➢ two-way NEST:

• 6.5km deterministic

• 20km ensemble

• Period: 01.09.2018 – 04.09.2018

• Lot more satellite data: SEVIRI, IASI

& ATMS humidity channels

Experimental setup (A) Experimental setup (B)


T [K] at 03.05.2016 15 UTC ~1000 hPa

22.01.2019


Assimilation Cycle

Experimental setup (A)


3. Numerical Testing – (A)

22.01.2019

RMSE

LETKF

LMCPF

RMSE

p-level [h

Pa]

Global RMSE for obs-fg statistics (Radiosondes vs. Model)

Period: 10.05.2016 – 31.05.2016

Relative humidityTemperature

~3%

~7%

~4%

~2% better



22.01.2019

RMSE

LETKF

LMCPF

RMSE

Channel num

ber

Global RMSE for obs-fg statistics

Period: 10.05.2016 – 31.05.2016

Bending Angles (GPSRO)𝑇𝐵𝑟𝑖𝑔ℎ𝑡𝑛𝑒𝑠𝑠 (IASI)

~6%~15%

Heig

ht

[m]

~17%

~3%


Assimilation Cycle

Experimental setup (B)


3. Numerical Testing – (B)

22.01.2019

RMSE

LETKF

LMCPF

RMSE

p-level [h

Pa]

Global RMSE for obs-fg statistics (Radiosondes vs. Model)

Period: 01.09.2018 – 04.09.2018

Relative humidityTemperature

~3%

~2%

~3% better


3. Numerical Testing – (B)

22.01.2019

RMSE

LETKF

LMCPF

RMSE

Channel num

ber

Global RMSE for obs-fg statistics

Period: 01.09.2018 – 04.09.2018

AIREP (WIND)IASI (𝑇𝐵𝑟𝑖𝑔ℎ𝑡𝑛𝑒𝑠𝑠)

~2%

~2% p-level [h

Pa]

~1% better

~6%



22.01.2019

mean

min

max

LETKF

LAPF

LMCPF

Global spread of T [K] ~ 500 hPa


Forecast Cycle



22.01.2019

Global verification of different lead times against Radiosondes

Period: 10.05.2016 – 24.05.2016


4. Summary

• Two localized particle filters working stably in an operational setup: LAPF and

LMCPF

• Strong improvements of scores by including model error via a Gaussian

mixture approach for the prior distribution

• LMCPF partly decouples spread and assimilation functionality (via model error)

• Particle Filter shows behavior similar to LETKF, with

some scores better than LETKF, some worse (o-b and forecast!)

• First particle filter tests with full global operational resolution and setup,

including two-way European nest in global DA

• LMCPF also works similarly for COSMO convective scale

22.01.2019


References

• Gerald Desroziers and Serguei Ivanov. “Diagnosis and adaptive tuning of observation-error

parameters in a variational assimilation”. Quarterly Journal of the Royal Meteorological Society,

2001

• Brian Hunt, Eric Kostelich and Istvan Szunyogh. “Efficient data assimilation for spatiotemporal

chaos: A local ensemble transform Kalman filter”. Physica D: Nonlinear Phenomena, 2007

• Gen Nakamura and Roland Potthast. “Inverse Modeling”. IOP Publishing, 2015

• Roland Potthast, Anne Walter and Andreas Rhodin. “A Localized Adaptive Particle Filter (LAPF)

within an operational NWP Framework”. MWR, 2019

• Sebastian Reich and Colin Cotter. “Probabilistic Forecasting and Bayesian Data Assimilation”.

Cambridge University Press, 2015

• Christoph Schraff, Hjendrick Reich, Andreas Rhodin, Annika Schomburh, Klaus Stehphan, Africa

Periáñez and Roland Potthast. “Kilometre-scale ensemble data assimilation for the cosmo model

(kenda)”. Quarterly Journal of the Royal Meteorological Society, 2016

• Peter Jan van Leeuwen, Yuan Cheng and Sebastian Reich. “Nonlinear Data Assimilation”. Frontiers

in Applied Dynamical Systems: Reviews and Tutorials. Springer, 2015

22.01.2019

Thanks for the attention!

Questions?

Appendix


Appendix

Global histogram of number of particles with weight ≥ 1

31.05.2016 21 UTC ~1000 hPa

22.01.2019


Appendix

31.05.2016 21 UTC ~ 100 hPa

31.05.2016 21 UTC ~ 500 hPa

31.05.2016 21 UTC ~1000 hPa

Global histograms of number of

particles with weight ≥ 1

22.01.2019


Global Model Verification Comparison

Created: 17.01.2019

EnVarICON

22.01.2019


Global Model Verification Comparison -

EPS

Created: 17.01.201922.01.2019

a localized markov chain particle filter (lmcpf) for the ... · a localized markov chain particle...

Documents