amdar (aircraft) and radar data assimilation ming hu 1,2, stan benjamin 2, william moninger 1,2,...
TRANSCRIPT
1
AMDAR (aircraft) and Radar Data Assimilation
Ming Hu1,2, Stan Benjamin2, William Moninger1,2, Curtis Alexander1,2, Stephen Weygandt2, David Dowell2 , Eric James1,2
1CIRES, University of Colorado at Boulder, CO, USA, 2 NOAA/ESRL/GSD/AMB, Boulder, CO, USA
Scientific Conferences of WMO–RA III Meeting
18 September 2014 - Asuncion, Paraguay
Thanks to Shun Liu and Jacob Carley from NCEP for contributions in radar data assimilation and process
2
What is Data Assimilation• Numerical Weather Prediction (NWP) is an initial-condition
problem– Given an estimate of the present state of the atmosphere (initial
conditions), and appropriate surface and lateral boundary conditions, the model simulates (forecasts) the atmospheric evolution
– The more accurate the estimate of the initial conditions, the better the quality of the forecasts (“accurate” doesn’t mean to fit to the observations very close)
• Data Assimilation: The process of combining observations and short-range forecasts to obtain an initial condition for NWP
• The purpose of data assimilation is to determine as accurately as possible the state of the atmospheric flow by using all available information for NWP
3
Data Assimilation: Variational Method (VAR)
• J is called the cost function of the analysis (penalty function) – Jb is the background term
– Jo is the observation term
• The dimension of the model state is n and the dimension of the observation vector is p:
– xt true model state (dimension n)
– xb background model state (dimension n)
– xa analysis model state (dimension n)
– y vector of observations (dimension p) – H observation operator (from dimension n to p)
– B covariance matrix of the background errors (xb – xt) (dimension n n)
– R covariance matrix of observation errors (y – H[xt]) (dimension p p)
J(x) = (x-x b)TB - 1(x-x b)+(y-H[x])TR - 1(y-H[x])= J b + J o
In this talk, we will use variational method to explain AMDAR and Radar data assimilation but most of steps are same for Ensemble based data analysis method
4
VAR: Background Term
• Background (forecast field): xb
• Analysis: x– Start from x=x b
• Analysis increment: x-xb
• Background error covariance: B– Variance: the background quality – Correlation
• Horizontal and vertical relation between 2 analysis point• Balance: relation between two analysis variables
J (x ) = (x-x b ) T B - 1 (x-x b )+(y-H[x ] ) T R - 1 (y-H[x ] )
5
VAR: Observation Term
• Observation: y• Observation operator: H[x]
– Conventional observation: • T, wind, moisture, Ps• 3D interpolation
– Non-conventional observations:• Radiance, Radar, GPSRO, …• Complex function
• Observation innovation: y-H[x]• Observation error variance: R
– Assumption: No correlation between two observations
J (x ) = (x-x b ) T B - 1 (x-x b )+ (y-H[x] ) T R - 1 (y-H[x] )
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
y
y
6
VAR: Steps of using observations
Step 1: Understand observations Y• Name• Variables observed• Geographic and time distribution • Space and time resolution• Observation errors• Quality Control and bias correction• Acceptable format (BUFR,…)• …
J (x ) = (x-x b ) T B - 1 (x-x b )+(y-H[x] ) T R - 1 (y-H[x] )
Step 2: Build observation operator:link analysis variables x to observations H[x]• Conventional observations:
– AMDAR observations– 3D interpolation
• Non conventional observations: Complex function– Radar Reflectivity = f(qr, qs, qh)– Radar radial wind=f(u, v, w)
Step 3: Define observation error variance R
1 2 3 1 2Background term are the same for all observations
7
AMDAR data assimilation• Understand AMDAR (commercial aircraft) Data
– What is AMDAR– What is observed– Data coverage and resolution– Observation errors– Quality control and bias correction
• Use AMDAR data in data assimilation• The impact of the AMDAR data
– Regional NWP system (Rapid Refresh for US/NCEP)– Global NWP system (ECMWF)
8
Understand AMDAR Data• AMDAR: Aircraft Meteorological Data Relay
• AMDAR is the automated measurement and transmission of meteorological data from an aircraft platform
• The AMDAR observing system is now recognized by WMO as a critical component of the WMO Global Observing System(GOS)
9
What is observed in AMDAR• Vertical profiles are derived as the aircraft is on ascent or descent • en-routed data are derived at cruise altitudes of around 35,000 feet (10,500 meters).• The meteorological parameters can be measured or derived:
1. Air temperature (static air temperature) 2. Wind speed and direction3. Pressure altitude (barometric pressure)4. Turbulence (Eddy Dissipation Rate or Derived Equivalent Vertical Gust)
• Additional non-meteorological parameters include:1. Latitude position 2. Longitude3. Time4. Icing indication (accreting or not accreting)5. Departure and destination airport6. Aircraft roll angle7. Flight number
• A water vapor (humidity) measurement can also be derived– Water vapor sensor: The Water Vapor Sensing System 2nd Generation (WVSS-II)– Operationally in the USA and Europe
Temperature , Wind, Pressure
Location, time, flight number
Humidity
10
List of AMDAR Participating Airlines
• As of July 2013, more than 400,000 observations per day world wide• 2863 aircraft are currently reporting world-wide, not including
Aeromexico data (coming soon) since 1-Aug-2014
11
24h AMDAR Data Coverage
Typically, every 5-10 minutes of regular real-time reports of meteorological variables whilst en-route at cruise level
12
AMDAR Vertical profiles
• High resolution vertical profiles: High-reso profiles are reported every 300 feet in low level and every 1000 feet in middle and upper
• Number vary a lot by airport:• Busy airport during the day : every 15 min or less• Many: couple profiles per day
Details in http://amdar.noaa.gov/new_soundings/2014_08_31.html
LAS VegasAscent Sounding
Montreal Ascent Sounding (No moisture)
DenverDescent Sounding
13
AMDAR data coverage over US
18 – 21 afternoon 21 – 00 early night
00 – 03 night 03 – 06 early morning 06 – 09 morning
09 – 12 day 12 – 15 day
15 – 18 day
Time in plots is US mountain time.
Big various during day and night
Obs
Num
ber
14
AMDAR data coverage over S. America
20-23 23-02 02-05 05-08
08-11 11-14 14-17
17-20
Time in plots is local time in Asuncion, Paraguay
Most of observations are during the night time
During the day time
Late afternoon to early night night Early morning
15
Data Quality• The expected uncertainties for basic AMDAR data
parameters (based on AMDAR Reference Manual):Variable UncertaintyTemperature +/- 1.0 CWind Vector +/- 2-3 m/sPressure Altitude +/- 4 hPa
AMDAR observation error used by GSI for US NAM and RAP applications
• Data analysis uses tuned observation errors
Pressure (hPa) Temperature (C) Wind (m/s) moisture (RH %)
700 0.87349 2.7272 16.605
500 0.74080 2.6219 17.791
200 0.93292 2.3213 19.748
16
AMDAR data quality control• GSD AMDAR Rejection list:
– Week-long statistics– Used by us to generate daily
aircraft reject lists for RAP and 3km HRRR (GSD/NOAA regional models)
– Used actively by NWS and other centers
Statistics of aircraft obs against Rapid Refresh 1-h forecasts:
bias_T > 2 (°C), std_T > 2 (°C),bias_S > 2 (m/s), std_S > 5 (m/s),bias_DIR > 7°, std_DIR > 30°,std_W > 5(m/s), rms_W > 7(m/s),bias_RH > 10%, std_RH > 20%,
;tail errors FSL MDCRS N bs_T Std_T bs_S std_S bs_D std_D std_W rms_W bs_RH std_RH (failures)
00000400 T W R 0 -------- 0 0.0 0.0 0.0 0.0 0 0 0.0 0.0 0.0 0.0 UND_Piper ( resrch_T_W_R )00000450 T W R 0 -------- 0 0.0 0.0 0.0 0.0 0 0 0.0 0.0 0.0 0.0 UND_Piper ( resrch_T_W_R )EU8001 - W - 10081 10081 409 -0.5 0.9 0.5 6.1 -1 15 7.4 9.1 0.0 0.0 Unknown ( std_S std_W rms_W )N194AA - W - 2777 2777 483 0.2 0.7 0.4 1.6 -7 14 1.2 2.3 0.0 0.0 Unknown ( bias_DIR )N203WN T - - 11060 11060 3085 2.6 1.2 0.2 2.4 1 8 1.8 3.5 -7.6 13.5 Unknown ( bias_T )N220WN T - - 11002 11002 3268 2.7 1.3 0.3 2.2 1 11 1.8 3.4 -9.2 16.7 Unknown ( bias_T )N402WN - - R 11149 11149 2853 -0.3 0.8 0.1 2.3 1 10 1.8 3.4 6.5 29.1 Unknown ( std_RH )N407WN - - R 11087 11087 2459 -0.4 1.3 1.0 2.6 -1 10 2.2 4.0 0.4 21.6 Unknown ( std_RH )N421LV T - - 10973 10973 2948 3.5 1.2 0.3 2.0 -1 7 1.4 2.9 0.0 0.0 Unknown ( bias_T )
Example of GSD AMDAR rejection list:
Aircraft tail number
Which observed variables to reject
17
GSD Aircraft Rejection ListVector wind difference between
AMDAR observations and RR 1h forecasts.
Shows some obvious bad aircraft winds near Chicago.(Used by NWS to help identify bad aircraft data.)
18
Bias of AMDAR obs and bias correction• Comparing with RAOBs, AMDAR temperature observations
are biased:• At flight levels, AMDAR have a warm bias. (BUT RAOBs have a cold bias at
the same level (ref 1). So exactly how to balance these two biases to move the model temperature toward the truth is a difficult --and somewhat philosophical--question)
• Bias can be dependent to fly phase (ref 2):– Descent: a strong cool bias– Ascent: a slight warm bias
• Bias correction (BC) to the AMDAR observations– ECMWF: variational aircraft temperature BC– NCEP: testing similar method as ECMWF with GSI– NASA/GMAO: bias calculated after analysis and var method
1.Bomin Sun, Anthony Reale, Steven Schroeder, Dian J. Seidel and Bradley Ballish (2013) Toward improved corrections for radiation-induced biases in radiosonde temperature observations. JGR 118, 4231-4243.2.Barry Schwartz and Stanley G. Benjamin, 1995: A Comparison of Temperature and Wind Measurements from ACARS- Equipped Aircraft and Rawinsondes. Wea. Forecasting, 10, 528–544.
19
Use AMDAR data in analysis• Observation operator for AMDAR data:
– Conventional observations: T, Q, U, V– 3D interpolation from grid analysis field to
observation location– Same as other conventional observation, such as
sounding and surface observations• Better to define a data type for AMDAR, for
example, in NCEP PrepBUFR:
20
Use AMDAR data in analysis• Convert data to the values and format analysis
system accepts. Use GSI as an example: 1. Convert AMDAR data into temperature, U and V
component of wind observation, specific humidity2. Encode the observations into PrepBUFR file with all
other conventional observations.• Let analysis system run … • After analysis, must check the analysis impact:
– How many observations are used in the analysis– The distribution of the analysis increment over
analysis domain in one analysis– Data impact from OSE, …
21
AMDAR data impact: RAP• The Rapid Refresh (RAP) is a US operational hourly
updated regional numerical weather prediction system for aviation and severe weather forecasting.
Configuration:• 13 km horizontal North American grid• Twice daily partial cycles from GFS
atmospheric fields• Hourly continue cycled land-surface fieldsModel:• WRF-ARW dynamic coreData Assimilation:• GSI 3D-VAR/GFS-ensemble hybrid data
assimilation• GSI non-variational cloud/precipitation
hydrometeor (HM) analysis • Diabatic Digital Filter Initialization (DDFI)
using hourly radar reflectivity observation
RAP version 1 operational implementation: 01 May 2012RAP version 2 operational implementation: 24 February 2014
22
Observations used in RAPHourly Observations (2012)
Rawinsonde (T,V,RH)
Profiler – NOAA Network (V)
Profiler – 915 MHz (V, Tv)
Radar – VAD (V)
Radar reflectivity - CONUS
Lightning (proxy reflectivity)
Aircraft (V,T)
Aircraft - WVSS (RH)
Surface/METAR (T,Td,V,ps,cloud, vis, wx)
Buoys/ships (V, ps)
Mesonet (T, Td, V, ps)
GOES AMVs (V)
AMSU/HIRS/MHS radiances
GOES cloud-top pressure/temp
GPS – Precipitable water
WindSat scatterometer
23
Impact of AMDAR moisture in RAP
RH – national – 1000-400 hPa
#1 obs type = aircraftDistant #2 tie – Surface, GPS, raobs
RAP 2013
Valid 00z - daytime
Valid 12z - nighttime
12z and 00z combined
24
Impact of AMDAR Temp in RAP
Valid 00z - daytime
Valid 12z - nighttime
12z and 00z combined
national - 1000-100 hPa flight levels - 300-100 hPa
#1 = Aircraft#2 = RAOBs, surfaceAircraft more impact at 3h, more at flight levels
25
Impact of AMDAR wind in RAP
Wind - national – 300-100#1 = Aircraft#2 = RAOBs, Some impact from sfc, GPS-Met, AMVs
Valid 00z - daytime
Valid 12z - nighttime
12z and 00z combined
RAP Domain US CONUS region
26
AMDAR data impact (ECMWF)
From The Benefits of AMDAR to Meteorology and Aviation (WIGOS Technical Report 2014-1, Version 1, Jan 2014) Available one line: https://docs.google.com/file/d/0BwdvoC9AeWjUbG1MRlAyU0dhZEk/edit
27
Radar data assimilation• Understand Radar Data
– What is weather Radar – What is observed– Data coverage and resolution from single radar– Observation errors– Radar data Quality Control
• Radar observation operator• Assimilate radar reflectivity data with DDFI
and the data impact in RAP
28
What is weather radar• Weather radar, also called weather surveillance
radar (WSR) and Doppler weather radar, is a type of radar used to locate precipitation, calculate its motion, and estimate it type (rain, snow, hail etc.)
• Modern weather radars are mostly pulse-Doppler radar, capable of detecting the motion of rain droplets (radial velocity) in addition to the intensity of the precipitation (reflectivity)
• Both types of the data can be analyzed to determine the structure of storms and their potential to cause the severe weather
29
Radar Coordinate and ResolutionRadar coordinate:The radar is located at the origin of the coordinate system; the Earth’s surface lies in the x-y plane:Azimuth angle Θ ; Elevation angle α; Range R.
WSR88D data resolution:• Beam width: 1 degree• Scan levels (tilt): 14 scans • Maximum range: 230 km • Gate resolution:
• 250 m for Radial wind• 1 km for reflectivity
• Products update every 6 minutes
α
30
Understand 3D radar coverage
3 dimensional radar coverage from ground levelMaury Markowitz - Own work
The approximate height and width of the radar beam with distance from the radar site.
Values in red represent the different elevation angles in this VCP
31
US Radar Coverage
One of the only networks of fairly comprehensive, storm-scale data
reflectivity
Radial velocity Courtesy of Jacob Carley from NCEP
32
Radar Data Quality Control at NCEP• To meet the high standard required by data
assimilation, it is necessary to develop simple and efficient QC technique for operational applications.
• Radar data quality control is a necessary and initial step for operational applications of radar data.
• Develop statistically reliable QC techniques for automated detection of QC problems in operational environments
• Among various of radar data quality problems, radar measured velocities can be very different (≥10 m/s) from the air velocities in the presence of migrating birds.
Courtesy of Shun Liu from NCEP
33
Radar data QC at NCEP
Courtesy of Shun Liu; more details see Shun Liu’s talk at GSI tutorial 2010: http://www.dtcenter.org/com-GSI/users/docs/presentations_2010.phpOr Shun Liu, et al: WRS-88D Radar Data Processing at NCEP. Submitted to Wea. Forecasting.
Scatter plots of radial wind
Before QC
after QC
Composite reflectivity Before QC
after QC
composite dual-pol variable CC
34
Radial Velocity Operator
• No term for the vertical velocity (w) in GSI• Suggest only radial velocity observations
from lower elevations should be considered– Avoid contamination of the horizontal
wind field, especially due to hydrometeor sedimentation
Vr(θ, α)=u cosα cosθ + v cosα sinθ +[w sinα]
Elevation angle α 90° - azimuth angle θ
Wind components from background
Courtesy of Shun Liu and Jacob Carley from NCEP
35
Reflectivity Operator (example)
T <= 0 °C (dry snow)
T > 0 °C (wet snow)
rain hail
snow
From model forecast rain, snow, hail mixing ratio: qr, qh, qs
ToDavid C. Dowell, Louis J. Wicker, and Chris Snyder, 2011: Ensemble Kalman Filter Assimilation of Radar Observations of the 8 May 2003 Oklahoma City Supercell: Influences of Reflectivity Observations on Storm-Scale Analyses. Mon. Wea. Rev., 139, 272–294.
36
Example of Radial Velocity Data analysis with GSI
Full wind vectors Wind analysis Increment
Low row:Analysis using a 1/4th of the default decorrelation length
Upper row:Analysis using default decorrelation length
Radar reflectivity at 0900 UTC on 23 May 2005
Need to modify Background Error Matrix to improve Rv Courtesy of Shun Liu from NCEP
37
Reflectivity data assimilation in RAP/HRRR
Radar reflectivity data are assimilated through Diabatic digital filter initialization (DDFI)
Forward integration,full physics with radar-based latent heating
-20 min -10 min Initial +10 min + 20 min
Backwards integration, no physics
Initial fields with improved balance, storm-scale circulation
+ RUC/RAP Convection suppression
RUC / RAP/ HRRR model forecast
38
Impact of DDFI with reflectivity Low-level
Convergence
K=4 U-comp. diff (radar - norad)
K=17 U-comp. diff
(radar - norad)
NSSL radar reflectivity
(dBZ)
14z 22 Oct 2008Z = 3 km
Upper-levelDivergence
39
Eastern US, Reflectivity > 25 dBZ11-21 August 2011
• 3km HRRR forecasts improve upon RAP 13km forecasts, especially at coarser scales much better upscaled skill
• Radar DDFI adds skill at both 13km and 3km
CSI 13 km CSI 40 km
HRRR no radarRAP no radar
HRRR radarRAP radar
HRRR no radarRAP no radar
HRRR radarRAP radar
Radar Reflectivity Verification
40
HRRR Radar Reflectivity Assimilation
3-km Interp
Refl Obs
1 hr pre-fcst Obs
GSI HM Anx
GSI 3D-VAR
HM
Obs
15 hr fcst
13z 14z 15z
3 kmHRRRRun
15 hr fcst 3-km Interp
15 hr fcst 3-km Interp
15 hr fcst
3-km Interp
HRRR 2013 introduction of
3-km data assimilation (DA)On 10 April 2013
Latency reduced 45 min to 1-2 hrs
DigitalFilter
DigitalFilter
DigitalFilter
DigitalFilter
20
09
-12
20
13
41
HRRR 2013 Pre-forecast HourTemperature Tendency (i.e. Latent Heating) = f(Observed Reflectivity)LH specified from reflectivity observations applied in four 15-min periodsNO digital filtering at 3-kmReflectivity observations used to specify latent heating in previous 15-min period as follows:• Positive heating rate where obs reflectivity ≥ 35 dBZ over depth ≥ 200 mb (avoids bright banding)• Zero heating rate where obs reflectivity ≤ 0 dBZ• Model microphysics heating rate preserved elsewhere
LH = Latent Heating Rate (K/s)p = PressureLv = Latent heat of vaporizationLf = Latent heat of fusionRd = Dry gas constantcp = Specific heat of dry air at constant pf[Ze] = Reflectivity factor converted to rain/snow condensatet = Time period of condensate formation (300s i.e. 5 min)
-45
-30
-15
0
-60 -45 -30 -15 0Model Pre-Forecast Time (min)
42
HRRR 3-km Reflectivity Assimilation
Radar Obs05z 18 May 2013
3-km radar DANO 3-km radar
DA
05z + 45min
45 min fcst
0-hr fcst0-hr fcst
45 min fcst
43
HRRR Retrospective Statistical Verification
Optimal Bias = 1.0
With 3-km DAWithout 3-km
DA
With 3-km DAWithout 3-km
DA
Statistical Retrospective Comparison30 May - 04 June 2012 (55 matched runs)
3-km grid ≥ 35 dBZ Eastern US
Improved 0-2 hr convection
44
Real-time HRRR case study
HRRR 13 UTC run 8 hr forecast valid 21 UTC
HRRR CompositeReflectivity
ObservedReflectivity
Moore, OK 20 May 201320 – 21 UTC Tornadic Supercell