Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems
Yanqiu Zhu
Contributions from: John Derber, Andrew Collard, Dick Dee, Russ
Treadon, George Gayno, Jim Jung, Emily Liu, Min-Jeong
Part I: The enhanced radiance bias correction scheme
Part II: New emissivity predictor to handle large land-sea differences with the new CRTM (will be presented at the 6th WMO Symposium on Data Assimilation)
Part III: Radiance bias correction for all-sky radiance bias correction
Part I: Enhanced radiance bias correction scheme
The original radiance bias correction scheme
The motives to enhance the original scheme – the goals for the enhanced scheme
Methodology for the enhanced scheme
Experiment setup and results
Conclusions
Yanqiu Zhu John Derber, Andrew Collard, Dick Dee, Russ Treadon, George Gayno, Jim Jung
Why radiance bias correction – to generate unbiased analysis
• the error associated with the instrument and calibration
• the error of the radiative transfer model
• Inconsistencies among the assumptions of data usage and model as well as algorithms
Derber et al. (1991), Harris and Kelly (2001), Dee (2004), etc
4
The control variables and are estimated by minimizing (Derber et al., 1991, Derber and Wu, 1998)
),(~
),(~
2
1
)()(2
1)()(
2
1),(
1
11
xhyRxhy
BxxBxxxJ
T
bT
bbxT
b
is radiative transfer model )(xh
x
Air-mass dependent in GSI
Separate scan-angle dependent
Original radiance bias correction scheme
A two-step procedure
,),()(~ angleair bxbxhh
where )(),(1
xpxb ii
N
ii
air
is pre-specified parameter
x
5
is predictor term with predictor coeff. ip i
i
What we expect to achieve with the enhanced radiance bias correction scheme
Combine the scan angle and air-mass bias components inside the GSI variational framework. Eliminate the duplications and potential risk of opposite drifting
between the two components
Remove the pre-specified parameter for each predictor, so avoid the trial-and-error procedure for any new bias predictors
Apply a new pre-conditioning to the predictor coefficients taking into account of observation contribution, speeding up the convergence
6
What we expect to achieve with the enhanced radiance bias correction scheme (cont.)
Radiance bias correction is based on the data passing the quality control
Reasonable radiance bias correction estimate need to be manually derived prior to the use of any new radiance data or during the re-analysis process with the original scheme
Quality control is performed on the bias corrected data
7
What we expect to achieve with the enhanced radiance bias correction scheme (cont.)
Automatically detect any new/missing/recovery of radiance data and initialize new radiance data; Quickly capture any changes in the data and the system.
A new approach of specifying the background error variances for the predictor coefficients
An new initialization procedure for any new radiance dataAdd capability of bias correction for passive channels
We can use any new radiance data with initial radiance bias correction set to be zero 8
• is computed outside GSI, a running average of OMF
• The predictors of air-mass are computed inside the GSI
Enhanced scheme: a one-step procedure
Original scheme: a two-step procedureangleb
kK
kkN
angleb
1
• Variational angle bias correction, is computed inside the GSI along with other predictor terms
• satang file no longer needed
• Special attention to the initialization of bias coefficients: using quality-controlled data from previous run; or mode of all data
angleb
9
Change of the Preconditioning
• Hessian w.r.t. predictor coeff. of the cost function
aa ββ
ββ1T
β1
β
ββ2
2
βHHRHB
β
hJ
~ ,
Modified block-diagonal preconditioning is set to be the inverse of (Dee 2004)
Current preconditioner Modified preconditioner
βM
β
x
B
BB
0
0
β
x
M
BB
0
0
2
2
β J
For simplicity, only diagonal elements of are considered at each analysis cycle
βM
10
Currently scheme
Nii
,,1
,0.102
Set to be the estimate of analysis error variance of the coeff. from previous analysis cycle
New data: is initialized to be 10000.0
Missing data: is twice as much as that of the previous analysis cycle
Enhanced scheme
),,( 22
1 NdiagB
2
i
2
i
2
iAutomatically adjust variances of coeff.
More & high quality data
Smaller variance
Background variance for predictor coeff.
11
Bias correction for passive channels (passive_bc=.true.)
• Make preparation for assimilating passive channels data
• Aid quality control of other channels
),(~
),(~
2
1)()(
2
1)( 11 a
T
abT
b xhyRxhyBF
At the end of analysis, minimizing the functional
angleai
N
iiaa bxpxhxh
)()( ),(~
1
where
)()( ),(~
1ai
KN
iiaa xpxhxh
or If adp_anglebc=.t.12
Experiment configuration
• High Resolution: T574L64 3DVAR and T254L64 EnKF with 80 ensemble members
• 0.25 for static B and 0.75 for ensemble• Two-way hybrid coupling• EnKF relies on the GSI to perform the radiance bias
correction
CTL: prda141b, current radiance bias correction
RBC : prda141br, enhanced radiance bias correction
Experiment period: 00Z July 5, 2012 to 00Z Aug. 16, 2012
Spin-up period: 00ZJuly 5, 2012 to 18Z July 13, 2012 13
Conclusions
• One-step bias correction procedure
• Improved convergence rate
• Made it easier to add new bias predictors
• Added the capability to automatically detect any new/missing/recovery of radiance data
• New radiance data can be used without providing prior bias correction estimate
• Neutral forecast skills in NH and Tropics, better in SH
• The scheme was also tested on NDAS/NAM
• Currently is used in the wind energy project (Jacob’s talk at 6th WMO Symposium on Data Assimilation)
23
Part III: Radiance bias correction with all-sky radiance assimilation
Yanqiu Zhu, Emily Liu, Min-Jeong Kim, Andrew Collard, John Derber
Two other talks on all-sky radiance assimilation at NCEP: Emily Liu’s and Min-Jeong Kim’s talks
Bias correction for Operational clear radiance
• Angle bias correction
• Air-mass bias correction with five predictor terms: global offset, a zenith angle term, cloud liquid water, lapse rate, and the square of the lapse rate.
All-sky radiance assimilation
• Control variable: total cw/normalized cw/normalized RH
• State variables: Hydrometeors ql, qi, (qr, qs, qg, qh)
• The cloud liquid water predictor is removed
• Observation error based on averaged CLW
• Quality control: cloud filtering is removed
• Linearized moist physics
Issues of radiance bias correction with all-sky radiance assimilation
• Different error characteristics between clear-sky and cloudy radiance
• Different cloud information between radiance observation and first guess
1. O:clear vs. F:clear
2. O:clear vs. F:cloudy eliminate cloud
3. O:cloudy vs. F:clear add cloud
4. O:cloudy vs. F:cloudy
Ideally, Gaussian distributed (O:clr,F:clr) & (O:cld,F:cld), with one hump ((O:clr,F:cld), (O:cld,F:clr)) on each side
Issues of radiance bias correction with all-sky radiance assimilation (cont.)
OmF before BC OmF after BC
Lots of (O:cld, F:cld), too fewer other categories
Over bias corrected (O:cld,F:cld) and (O:clr,F:clr)
Issues of radiance bias correction with all-sky radiance assimilation (cont.)
OmF before BC OmF after BC
METOPA AMSUA Ch. 15
Issues of radiance bias correction with all-sky radiance assimilation (cont.)
METOPA AMSUA CH. 15
OmF before BC OmF after BC
(O:cld, F:cld)
• What may have hurt the radiance bias correction
• Experiments are underway, and results will be presented later at the GSI meeting
1. Bias correction coeff. are retrieved using only (O:clr,F:clr) and (O:cld,F:cld); Fixed previous coeff. are used to calculate the bias corrected OmF for (O:clr,F:cld) and (O:cld,F:clr).
2. Variational quality control
1. Inclusion of the (O:clr,F:cld) and (O:cld,F:clr) in bias correction
2. Radiance data with large OmF but small obs error
Experiment strategies: