convective-scale data assimilation in the weather research...
TRANSCRIPT
1/20
Convective-scale data assimilation in theWeather Research and Forecasting model
using a nonlinear ensemble filter
Jon Poterjoy, Ryan Sobash, and Jeffrey Anderson
National Center for Atmospheric Research†
ASP/MMM/DAReS
25th May 2016
†The National Center for Atmospheric Research is sponsored by the National Science Foundation.
2/20
Data assimilation in weather models
Filters and smoothers are applied regularly for dataassimilation in geophysics.
Current methods are based on variational approaches (3DVarand 4DVar), Ensemble Kalman filters (EnKFs), orcombinations of the two.
Assumptions:
The model dynamics are linear.
Observations relate linearly to the model state variables.
The model state and observation errors are Gaussian.
3/20
Example problem
100
150
200
250
300
*
Dis
tance (
km
)
140 160 180 200 220 240
5
10
15
Distance (km)
Heig
ht (k
m)
10 20 30 40 50 60
*
Given:
100-member ensemble forecastsare samples from prior errordistribution p(x)
Radar reflectivity measurement at? (denoted y).
Top and bottom panels show crosssections through true storm atobservation location.
Reflectivity and storm-relative windsare plotted.
4/20
Prior ensemble at ?
0
5
10x 10
−4
|y
qic
e
(g k
g−
1)
2
4
6
8
|y
qra
in
(g k
g−
1)
0
0.1
0.2
|y
qsn
ow
(g k
g−
1)
35 40 45 50 55
0
2
4
Reflectivity (dBZ)
|y
qg
rau
pe
l
(g k
g−
1)
|y
|y
|y
35 40 45 50 55Reflectivity (dBZ)
|y
p(x|y) ∝ p(y|x)p(x)
Blue markers: prior samples fromjoint probability distribution ofreflectivity and microphysicsvariables
Yellow markers: true state
Black tickmarks: observedreflectivity
5/20
EnKF update at ?
0
5
10x 10
−4
|y
qic
e
(g k
g−
1)
2
4
6
8
|y
qra
in
(g k
g−
1)
0
0.1
0.2
|y
qsn
ow
(g k
g−
1)
35 40 45 50 55
0
2
4
Reflectivity (dBZ)
|y
qg
rau
pe
l
(g k
g−
1)
|y
|y
|y
35 40 45 50 55Reflectivity (dBZ)
|y
p(x|y) ∝ p(y|x)p(x)
Blue markers: prior samples fromjoint probability distribution ofreflectivity and microphysicsvariables
Yellow markers: true state
Black tickmarks: observedreflectivity
Red markers: posterior samples
Green markers: posterior mean
6/20
Particle filter update at ?
0
5
10x 10
−4
|y
qic
e
(g k
g−
1)
2
4
6
8
|y
qra
in
(g k
g−
1)
0
0.1
0.2
|y
qsn
ow
(g k
g−
1)
35 40 45 50 55
0
2
4
Reflectivity (dBZ)
|y
qg
rau
pe
l
(g k
g−
1)
|y
|y
|y
35 40 45 50 55Reflectivity (dBZ)
|y
p(x|y) ∝ p(y|x)p(x)
Blue markers: prior samples fromjoint probability distribution ofreflectivity and microphysicsvariables
Yellow markers: true state
Black tickmarks: observedreflectivity
Red markers: posterior samples
Green markers: posterior mean
7/20
Particle filters as a practical solution?
Challenges:
PFs have known limitations for high-dimensional systems(e.g., Bengtsson et al. 2008; Bickel et al. 2008; Snyder et al.2008).
They may also be inappropriate for models containing errorsources that are represented poorly or ignored.
Several attempts have been made to circumvent these issues:van Leeuwen (2010), Frei and Kunsch (2013), Majda et al.(2014), Cheng and Reich (2015), etc.
8/20
The local PFPoterjoy (MWR, 2016) and Poterjoy and Anderson (MWR, 2016)introduce the local PF for data assimilation.
Local PF vs. EAKF (Anderson, 2001) using 40-variable Lorenz (1996) model:
Linearmeasurement
operator:H(x) = x
101
102
103
1
2
3
||
|||| || || || || ||
||
||
|
|
|| || || ||
Prio
r e
rro
r
Ensemble size
Average EAKF RMSEAverage PF RMSE
Nonlinearmeasurement
operator:H(x) = ln(|x |)
101
102
103
2
3
4
|
|| ||
|
|
||
| |
|
||
||
|
|
|||
| |
Prio
r e
rro
r
Ensemble size
Average EAKF RMSEAverage PF RMSE
9/20
Prior with ob located 2 km lower
0
5
10x 10
−4
|y
qic
e
(g k
g−
1)
2
4
6
8
|y
qra
in
(g k
g−
1)
0
0.1
0.2
|y
qsn
ow
(g k
g−
1)
35 40 45 50 55
0
2
4
Reflectivity (dBZ)
|y
qg
rau
pe
l
(g k
g−
1)
|y
|y
|y
35 40 45 50 55Reflectivity (dBZ)
|y
p(x|y) ∝ p(y|x)p(x)
Blue markers: prior samples fromjoint probability distribution ofreflectivity and microphysicsvariables
Yellow markers: true state
Black tickmarks: observedreflectivity
10/20
PF update
0
5
10x 10
−4
|y
qic
e
(g k
g−
1)
2
4
6
8
|y
qra
in
(g k
g−
1)
0
0.1
0.2
|y
qsn
ow
(g k
g−
1)
35 40 45 50 55
0
2
4
Reflectivity (dBZ)
|y
qg
rau
pe
l
(g k
g−
1)
|y
|y
|y
35 40 45 50 55Reflectivity (dBZ)
|y
p(x|y) ∝ p(y|x)p(x)
Blue markers: prior samples fromjoint probability distribution ofreflectivity and microphysicsvariables
Yellow markers: true state
Black tickmarks: observedreflectivity
Red markers: posterior samples
Green markers: posterior mean
11/20
Local PF update
0
5
10x 10
−4
|y
qic
e
(g k
g−
1)
2
4
6
8
|y
qra
in
(g k
g−
1)
0
0.1
0.2
|y
qsn
ow
(g k
g−
1)
35 40 45 50 55
0
2
4
Reflectivity (dBZ)
|y
qg
rau
pe
l
(g k
g−
1)
|y
|y
|y
35 40 45 50 55Reflectivity (dBZ)
|y
p(x|y) ∝ p(y|x)p(x)
Blue markers: prior samples fromjoint probability distribution ofreflectivity and microphysicsvariables
Yellow markers: true state
Black tickmarks: observedreflectivity
Red markers: posterior samples
Green markers: posterior mean
12/20
Case study: idealized MCS
Distance (km)
Dis
tan
ce
(km
)
a) 0 min.
50 100 150 200 250
50
100
150
200
250
300
350
Distance (km)
b) 30 min.
50 100 150 200 250
Distance (km)
c) 90 min.
100 150 200 250 300
Distance (km)
d) 180 min.
150 200 250 300 3500
10
20
30
40
50
60
Model: NCAR Weather Research and Forecasting model(3-km grid spacing with 40 vertical levels)Observations: radar velocity and reflectivity every 5 minutesData assimilation: Local PF and EAKF in DART frameworkusing 100 members
See Sobash and Stensrud (2013) for details
13/20
EAKF members (180 min)
Distance (km)
Dis
tance (
km
)
Truth
100 200 300
100
200
300
0
10
20
30
40
50
60
14/20
Local PF members (180 min)
Distance (km)
Dis
tance (
km
)
Truth
100 200 300
100
200
300
0
10
20
30
40
50
60
15/20
Probabilistic verification
0.02
0.04
0.06
0.08
qice
EAKF
Fre
quency q
ice
PF
0.02
0.04
0.06
0.08
qrain
Fre
quency q
rain
0.02
0.04
0.06
0.08
qsnow
Fre
quency q
snow
0.02
0.04
0.06
0.08
qgraupel
Fre
quency q
graupel
20 40 60 80 100
0.02
0.04
0.06
0.08
qcloud
Rank
Fre
quency
20 40 60 80 100
qcloud
Rank
Rank histograms calculatedfrom prior members every10 min in convective cellswithin leading edge of squallline.
Grid points where truereflectivity > 0 dBZ areused for verification,assuming no spatial andtemporal correlationsbetween variables.
16/20
RMSE and bias (5-min cycling)
−2 0 2 4
2
4
6
8
10
12
He
igh
t (k
m)
m s−1
u
0 1 2 3
m s−1
v
0 1 2
m s−1
w
0 0.5 1K
T
−0.5 0 0.5 1 1.5
g kg−1
qvapor
−0.1 0 0.1 0.2
2
4
6
8
10
12
He
igh
t (k
m)
g kg−1
qcloud
−0.02 0 0.02 0.04
g kg−1
qice
−0.2 0 0.2 0.4
g kg−1
qrain
0 0.2 0.4
g kg−1
qsnow
EAKF RMSE
EAKF biasPF RMSE
PF bias
−0.5 0 0.5 1
g kg−1
qgraupel
Horizontal mean RMSEs (solid) and bias (dashed) from 60-minforecasts.
Values are averaged in vicinity of squall line.
17/20
RMSE and bias (20-min cycling)
−2 0 2 4
2
4
6
8
10
12
Heig
ht (k
m)
m s−1
u
0 1 2 3
m s−1
v
0 1 2
m s−1
w
0 0.5 1K
T
−0.5 0 0.5 1 1.5
g kg−1
qvapor
−0.1 0 0.1 0.2
2
4
6
8
10
12
Heig
ht (k
m)
g kg−1
qcloud
−0.02 0 0.02 0.04
g kg−1
qice
−0.2 0 0.2 0.4
g kg−1
qrain
0 0.2 0.4
g kg−1
qsnow
EAKF RMSE
EAKF biasPF RMSE
PF bias
−0.5 0 0.5 1
g kg−1
qgraupel
Horizontal mean RMSEs (solid) and bias (dashed) from 60-minforecasts.
Values are averaged in vicinity of squall line.
18/20
Forecast error evolution
Mean forecast RMSEs as afunction of time.
Values are averaged in vicinityof squall line from 20-min obsfrequency experiment.
Initial error growth in EAKFforecasts is much more rapidthan in PF forecasts.
0 10 20 30 40 50 602.5
3
3.5
4
m s
−1
u
EAKF RMSE
PF RMSE
0 10 20 30 40 50 601.5
2
2.5
3
m s
−1
v
0 10 20 30 40 50 601.4
1.5
1.6
1.7
1.8
1.9
m s
−1
w
0 10 20 30 40 50 600.8
0.85
0.9
0.95
1
K
T
0 10 20 30 40 50 600.52
0.54
0.56
0.58
0.6
g k
g−
1
qvapor
0 10 20 30 40 50 600.075
0.08
0.085
0.09
0.095
0.1
g k
g−
1
qcloud
0 10 20 30 40 50 600.012
0.013
0.014
0.015g k
g−
1
qice
0 10 20 30 40 50 600.1
0.12
0.14
0.16
0.18
g k
g−
1
qrain
0 10 20 30 40 50 600.1
0.12
0.14
0.16
0.18
Forecast time (min)
g k
g−
1
qsnow
0 10 20 30 40 50 600.4
0.5
0.6
0.7
0.8
Forecast time (min)g k
g−
1
qgraupel
19/20
Summary
Particle filters provide a means of assimilating observations forapplications that are difficult for linear/Gaussian filters.
A new data assimilation system is developed thatapproximates the particle filter within local neighborhoods ofobservations (Poterjoy 2016).
The local PF is computationally affordable—with a costcomparable to the NCAR DART EAKF.
Recent testing of the local PF in the WRF model provide anincentive to explore real applications.
20/20
ReferencesAnderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129,2884–2903.
Bengtsson, T. and P. Bickel and B. Li, 2008: Curse-of-dimensionality revisited: Collapse of the particle filter invery large scale systems. Probability and Statistics: Essays in Honor of David A. Freedman, D. Nolan and T.Speed, Eds., 2, 316–334.
Bickel, P. and B. Li and T. Bengtsson, 2008: Sharp failure rates for the bootstrap particle filter in highdimensions. Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, 3,318–329.
Cheng, Y. and S. Reich, 2015: A McKean optimal transportation perspective on Feynman-Kac formulae withapplication to data assimilation. Frontiers in Applied Dynamical Systems., URLhttp://arxiv.org/abs/1311.6300.
Frei, M. and H. R. Kunsch, 2013: Bridging the ensemble Kalman and particle filters. Biometrika, 1–20.
Majda, A. J., D. Qi, and T. P. Sapsis, 2014: Blended particle filters for large-dimensional chaotic dynamicalsystems. Proc. Natl. Acad. Sci. U.S.A., 111, 7511–7516.
Poterjoy, J., 2016: A localized particle filter for high-dimensional nonlinear systems. Mon. Wea. Rev., 144,59–76.
Poterjoy, J. and J. L. Anderson, 2016: Efficient assimilation of simulated observations in a high-dimensionalgeophysical system using a localized particle. Mon. Wea. Rev., accepted.
Snyder, C., T. Bengtsson, P. Bickel, and J. Anderson, 2008: Obstacles to High-Dimensional Particle Filtering.Monthly Weather Review, 136, 4629–4640.
Sobash, R. A. and D. J. Stensrud, 2013: The Impact of covariance localization for radar data on EnKFanalyses of a developing MCS: Observing system simulation experiments. Mon. Wea. Rev., 141, 3691–3709.
van Leeuwen P. J., 2010: Nonlinear data assimilation in geosciences: an extremely efficient particle filter.Quart. J. Roy. Meteor., 136, 1991–1999.