application of machine learning methods to model bias
TRANSCRIPT
2020.01.30 Data Assimilation Workshop
Application of machine learning methods to model bias correction:
Lorenz-96 model experiments
Data Assimilation Research Team, RIKEN Center for Computational Science
Arata Amemiya*, Takemasa Miyoshi
Shlok Mohta
Graduate School of Science, the University of Tokyo
Motivation: bias in weather and climate models
Weather and climate models have model biases from various sources
β’ Truncation errorβ’ Approximation of unresolved physical processes
- Convection- Small-scale topography- Turbulence- Cloud microphysics
(From JMA website)
Treatment of forecast error in data assimilation
π·π‘+1π
= π΄π·π‘ππ΄π»
Kalman filter
ππ‘+1π
=π(ππ‘π)
π΄ = ππ/ππ
http://www.data-assimilation.riken.jp/jp/research/index.html
π·π = π° β π²π― π·π
π² = π·ππ―π π―π·ππ―π + πΉβ1
ππ = ππ +π² π β π»(ππ)
Model bias leads tothe underestimation of forecast(background) error
Previous analysis
Analysis mean
Observation
Forecast mean
Update state and forecast error covariance
Calculate analysis state and error covariance
Calculate Kalman gain
Treatment of imperfect model
Insufficient model error degrades the performance of Kalman filter
1. Covariance inflation
2. Correction of systematic bias component
π·π β πΌ π·π
ππ‘+1π
= ππ‘+1π
+ π
ππ‘+1π = ππ‘+1
π+π² ππ‘+1 β π» ππ‘+1
π
Relaxation-to-prior
multiplicative inflation
π·π β π·π +πΈadditive inflation
π·π β (1 β πΌ) π·π + πΌπ·π
Bias correction with simple functional form
Dimensionality reduction can be applied using Singular Value Decomposition (SVD) (Danforth et al. 2007)
Set of training data {πΏπ, ππ}β bias correction term π«(π) estimation
π«0 = Ξ€πΏπ Ξπ‘
π³πβ² = πͺπΏπ,π πͺπ,πβ1πβ² β Ξπ‘
π« π = π«0 + π³πβ² πβ² = ππ β ππ
Steady component
πͺ : correlation matrix
Linearly-dependent component (Leith , 1978)
β βOfflineβ bias correction
π
ππ‘π = πtrue(π)
π
ππ‘π = πmodel π
πΏπ = ππ‘ π‘ + Ξπ‘ β ππ π‘ + Ξπ‘
ππ‘ π‘ , ππ‘ π‘ + Ξπ‘ β¦
ππ π‘ + Ξπ‘
π
ππ‘π = πmodel π + π«(π)
Simplest form: linear dependency
Bias correction with nonlinear basis functions
β’ Coupled Lorenz96 system (Wilks et al. 2005, Arnold et al. 2013)
β’ Real case: All-sky satellite infrared brightness temperature (Otkin et al. 2018)
Probability of (obs β fcst) vs obs
Higher order polynomials :
Neural networks :
β’ Coupled Lorenz96 system (Watson et al. 2019)
(Fig.2 of Otkin et al. 2018)
Online bias correction
β’ Kalman filter
= Simultaneous estimation of state variables and bias correction terms
- Polynomials (Pulido et al. 2018)
β βOnlineβ bias correction
- Legendre polynomials(Cameron and Bell, 2016; for Satellite sounding in UK Met Office operational model)
- Steady component (Dee and Da Sliva 1998, Baek et al 2006)
β’ Variational data assimilation (βVarBCβ)
sequential treatment / augmented state
Penalty function forbias-corrected obs error
Penalty function forregularization
Localization
Localization is built-in in LETKF
(Pathak et al. 2018, Watson et al. 2019)
Also in ML-based data driven modelling
β Reduced matrix size -> low cost
In high dimensional spatiotemporal system,geographically (and temporally) local interaction is usually dominant
β Highly effective parallelization(Miyoshi and Yamane, 2007)
Also used in simultaneous parameter estimation(Aksoy et al. 2006)
(Pathak et al. 2018)
The goal of this study
β ML-based online bias correction using Long-Short Term Memory (LSTM)
β Combined with LETKF with similar localization
Test experiments with coupled Lorenz96 model
β’ Experimental Setup
β’ Online bias correction with simple linear regression as a reference
β’ (Online bias correction with LSTM)
bias correction in LETKF system
Model
ππ‘+1π = ππ‘+1
π+π² ππ‘+1 β π» ππ‘+1
π
observationforecast
ππ‘+1π
Bias correction
Analysis
Bias correction function π(ππ) update
ππ‘+1 = π΄ ππ‘
ππ‘+1π
= π(ππ‘+1π
)
LETKF
ππ‘+1
Lorenz96 model
π
ππ‘π₯π = π₯πβ1 π₯π+1 β π₯πβ2 β π₯π + πΉ
π₯π π₯π+1
β’ 1-D cyclic domainβ’ Chaotic behavior for sufficiently large πΉ
Wind @ 250hPaNCEP GFS
https://earth.nullschool.net/
(Lorenz 1996)
π = 1,2,β¦ , πΎ
π₯ = πΉ
πΎ = 40, πΉ = 8
Largest Lyapunov exp.
Mean π₯ π‘
Coupled Lorenz96 model
π
ππ‘π₯π = π₯πβ1 π₯π+1 β π₯πβ2 β π₯π + πΉ β
βπ
π
π=π½ πβ1 +1
ππ½
π¦π
π
ππ‘π¦π = βππ π¦π+1 π¦π+2 β π¦πβ1 β ππ¦π +
βπ
ππ₯int[(πβ1)βπ½]+1
π = 1,2,β¦ , πΎ
π = 1,2,β¦ , πΎπ½
βNature runβ
π
ππ‘π₯π = π₯πβ1 π₯π+1 β π₯πβ2 β π₯π + πΉ + π΄ sin 2π
π
πΎ
Forecast model (Danforth and Kalnay, 2008)
Coupling terms
Multi-scale interaction
π¦1 π¦π½
Parameters used in this studyοΌ
(Wilks, 2005)
β¦ β¦
(πΎ = 8)
Large scale (Slow) variables
Small scale (fast) variables
πΎ = 16, π½ = 16
β = 1, π = 20, π = 50
πΉ = 16, A = 1
Data assimilation experiment
Observation operatorοΌidenticalοΌobs = model gridοΌ
Error standard deviationοΌ 0.1
Interval: 0.025 / 0.05 / 0.1 (cf: doubling time β 0.2)
MemberοΌ 20LocalizationοΌ Gaussian weighting (length scale = 3 grids)
Covariance inflation: multiplicative (factor: πΌ)
Obs interval Minimum RMSE Inflation πΌ
0.025 0.0760 2.9
0.05 0.0851 5.8
0.1 0.0902 15.0
βObservationβ
LETKF configuration
Large inflation factor πΌ is needed to stabilize DA cycle
βNature runβ + random error
βControl runβ: Data assimilation without bias correction
Example: simple linear regression
Model
ππ‘+1π = ππ‘+1
π+π² ππ‘+1 β π» ππ‘+1
π
ππ‘+1 = π» ππ‘+1true + πobs
observationforecast
ππ‘+1π
Bias correction
AnalysisLETKF
ππ‘+1π
= πΎπ+ππ
ππ‘+1π
+ ππ‘+1π
Online bias correction by linear regression
πΎπ‘+1π = πΎπ‘+1
π+ π½π² ππ‘+1 β π» ππ‘+1
πβ (ππ‘+1
π)π β ππ‘+1
π π
ππ‘+1 = π΄ ππ‘
ππ‘+1 = (1 β πΎ)ππ‘
πΎπ‘+1 = 1 β πΎ πΎπ‘
ππ‘+1π = ππ‘+1
π+ π½π² ππ‘+1 β π» ππ‘+1
π
Online bias correction by linear regression
π½ = 0.02, πΎ = 0.999 (half-life: ~37) ππ
ππ β ππ
ππ β ππ :correction
ππ β ππ has large negative correlation with ππ
Interval (day) Min. infl. RMSE
0.025 2.9 0.0760
0.1 15.0 0.0902
Interval (day) Min. infl. RMSE
0.025 2.0 0.0625
0.1 8.2 0.0849
and mostly offset by bias correction term
Stable with smaller inflation factor πΌ
Correction term
Time 0-30 120-150
240-270 360-390
Bias: Forecast(before correction)-analysisCorrection
Nonlinear bias correction with ML
Model
ππ‘+1π = ππ‘+1
π+π² ππ‘+1 β π» ππ‘+1
π
observationforecast
ππ‘+1π
Bias correction
Analysis
Train the network π with ππ‘+1π
, ππ‘+1π
ππ‘+1 = π΄ ππ‘
ππ‘+1π
= π(ππ‘+1π
, β¦ )
LETKF
ππ‘+1
RNN
Plan: LSTM implementation
β’ Activation:tanh / sigmoid(recurrent)
LSTM
β’ No regularization / dropout
ππ‘π, ππ‘
π
Input
ππ‘βΞπ‘π
β¦ππ‘π
nx=9, nt=5
Outputππ‘π
nx=1
LSTMnx=15
Densenx=10
Densenx=5
Lorenz96LETKF ππ‘
π
Python TensorFlowTensorflow LSTM is implemented and integrated with LETKF codes
Network architecture
β’ 1 LSTM + 3 Dense layers
β’ LETKF-like Localization
Input : localized area
Output : one grid point
Summary
β’ LSTM-based bias correction is implemented and to be tested
β’ The efficiency of localization ?
β’ Systematic model bias degrades forecasts and analysis
β’ βOnlineβ bias correction with data assimilation has been studied using a fixed basis function set
β’ βOfflineβ bias correction can be performed by ML as nonlinear regression
β’ Online learning ?