NUMERICAL WEATHER PREDICTION NUMERICAL WEATHER PREDICTION ((Data AssimilationData Assimilation Part) Part)

Dr Meral DemirtaşDr Meral DemirtaşTurkish State Meteorological ServiceTurkish State Meteorological Service

Weather Forecasting DepartmentWeather Forecasting Department

WMO, Training Course, 26-30 September 2011WMO, Training Course, 26-30 September 2011Alanya, TurkeyAlanya, Turkey

Outline

• Introduction

• Basic concepts

• Observation resources

• Data assimilation techniques

Introduction

NWP is an initial/boundary value problem.• Having the following conditions, the model simulates or

forecasts the evolution of the atmosphere.– an estimate of the present state of the atmosphere (initial conditions)– suitable surface and lateral boundary conditions

• The more accurate the estimate of the initial conditions, the better the quality of the forecasts.• Operational NWP centres produce initial conditions through a statistical combination of observations and short-range forecasts. This approach is called data assimilation.

The purpose of a data assimilation is to blend in observations originating from different sources with information contained in a prior estimate of the state of the atmosphere (background). The background is modified to incorporate new observations by combining new and old information in a statistically optimal way. Then, the role of an NWP model is to carry the information gained from past observations forward in time.

Data AssData Assiimmiilatlatiion (Don (DAA) Processes) Processes

Ingesting the data Decoding coded observations Weeding out bad data Comparing the data to first guess fields Interpolating the data on to the model grid for making the forecast

Quality control (QC)

• First guess based rejections

• Variational QC rejections

Compute increments

Analysis

Observational data pre-processing

Preliminary work:

• Check out duplicate reports• Hydrostatic check• Black-listing: Data skipped due to systematic bad performance • Thinning: Some data are not fully used to avoid over-sampling and correlated errors

Observation types Variables used Notes

SYNOP (synoptic) u,v,T,ps (or z), rh u/v, used only over sea, in the tropics also over low terrain (<150 m). Orographic rejection limit 6hPa for rh,100hPa for z and 800m for ps.

AIREP u,v,T Used only below 50hPa.

SATOBs u,v Selected areas and levels

DRIBU u,v,ps Orographic rejection limit 800 m for, ps.

TEMP u,v,T, q, rh q only below 300hPa. 10m u/v used over land only in tropics (<150m. 10hPa orographic rejection limit for u/v, 100 hPa for z/T, 6hPa for rh and 4hPa for q.

PILOT u,v 10m u/v used over land only in tropics over low terrain (<150 m). Orographic rejection limit 10hPa for u/v

SATEM Tb Selected channels and areas.

SCATT u,v Used if SST warmer than 273K or if both observed and background wind less than 25m/s.

Radar u,v,q Cluttering and de-aliasing issues…

SYNOP/METAR/SHIP: MSLP, 10m-u/v, 2m-rh.

TEMP: Wind, T, q

Aircraft: Wind, Temperature

Pilot/Profilers: Wind

DRIBU: MSL Pressure, Wind-10m

Remote sensing retrievals

• Atmospheric Motion Vectors (AMV) (geo/polar).• SATEM thickness.• Ground-based GPS Total Precipitable Water• SSM/I oceanic surface wind speed and TPW.• Scatterometer oceanic surface winds.• Radar radial velocity and reflectivity• Satellite temperature/humidity/thickness profiles.• GPS refractivity (e.g. COSMIC).• HIRS: NOAA-16, NOAA-17, NOAA-18, METOP-2• AMSU-A: NOAA-15, NOAA-16, NOAA-18, • AMSU-B: NOAA-15, NOAA-16, NOAA-17• AIRS: EOS-Aqua• SSMIS: DMSP-16• EOS-Aqua, METOP-2

Data coverage issues

• Primitive equation based models have a number of degrees of freedom in the order of 107.

• For a time window of +/-3 hours, there are typically10 to 100 thousand observations of the atmosphere. They are distributed non-uniformly in space and time. It is necessary to use additional information, called the background field, first guess or prior information.

• A short-range forecast is used as the first guess in operational data assimilation practices.

• Current operational systems typically use a 3/6-h cycle performed four times a day.

In the operational NWP practice, it is not sufficient to perform spatial interpolation of observations into regular grids. There are not enough data available to define the initial state. The number of degrees of freedom in an NWP model is of the order of 107, while the total number of conventional observations is of the order of 104–105.

There are many new types of data such as satellite and radar observations, but:

• they do not directly measure the variables used in the models• their distribution in space and time is very non-uniform.

In addition to observations, it is necessary to use a first guess estimate of the state of the atmosphere at the grid points.

The first guess (or background field) is our best estimate of the state of the atmosphere prior to the use of the observations.

A short-range forecast is normally used as a first guess in operational systems in what is called an analysis cycle.

• Over data rich regions, the analysis is dominated by the information contained in the observations.

• In data-poor regions, the forecast benefits from the information coming from the upstream.

For instance, 6-h forecasts over the North Atlantic Ocean are usually good, because of the information coming from the observation rich North America. The model is able to transport information from data-rich to data-poor areas.

Global analysis cycle. Regional analysis cycle.

Data Assimilation (DA) TechniquesData Assimilation (DA) Techniques

• Empirical Assimilation Methods• Successive Corrections-Iterative analysis (empirical)• Newtonian Relaxation (nudging)

• Sequential Methods • Optimal interpolation (OI) (statistical)• 3-Dimentional “VARiational” DA (statistical)

• Non-Sequential Methods• 4-Dimentional “VARiational” DA (statistical)• Ensemble Kalman Filter (advanced)

• Hybrid methods: • Combinations of ensemble DA techniques:

ETKF-3DVar, EnKF-4DVar (advanced)

Suppose we have the pressure, pi and temperature, Ti, every day for a year. Let n=365.

The mean pressure is:

and like-wise for mean temperature.

The variance of pressure:

and like-wise for temperature variance σ2T .

The standard deviations, σp and σT are the square roots of the variances. They measure the root mean square deviation from the mean.

Recalling Some Basics of Statistics

The covariance of p and T is defined as:

The correlation between p and T is the normalized covariance:

It is a dimensionless number, and bound between -1 and +1.

If p and T tend to be greater than their mean values at the same time, or less than their mean values at the same time, they are positively correlated and ρpT > 0.

If p tends to be greater than its mean value when T is less than its mean, and vice-versa, then p and T are negatively correlated and ρpT < 0.

• An assimilation system deals with: Observations (yo)

Background field (xb)

Observation and forecast errors and statistics

• An assimilation system generates an analysis (synthesis).

Analysis is used in a number of avenues: Initial conditions for NWP models Climatology and re-analyses Observing system design

Fundementals of a DA System

Say the back ground field is a model 6-h forecast: xb

If the observed quantities are not the same as the model variables, the model variables are converted to observed variables: yo

The first guess of the observations is denoted as: H(xb)where H is the observation operator.

The difference between the observations and the background: yo − H(xb)

It is referred as the observational increment or innovation.

Observations (yo)

First guess at the grid points (Xb)

First guess at the observation location H(Xb)

The innovation vector (yo-H(Xb))

The difference between observations and the first guess are taken at the observation location and referred as innovation vectors (yo-H(x)).

A schematic illustration of innovation vectors

The analysis xa is obtained by adding the innovations to the background field with weights W that are determined based on the estimated statistical error covariances of the forecast and the observations:

xa = xb +W[yo − H(xb)]

Different analysis schemes (SCM, OI, 3D-Var,and KF) are based on this equation, but di er by the approach taken to ffcombine the background and the observations to produce the analysis.

Earlier methods such as the SCM used weights which were determined empirically. The weights were a function of the distance between the observation and the grid point, and the analysis was iterated several times.

• In Optimal Interpolation (OI), the matrix of weights W is determined from the minimization of the analysis errors at each grid point.

• In the 3D-Var approach, one defines a cost function proportional to the square of the distance between the analysis and both the background and the observations. This cost function is minimized to obtain the analysis.

• Lorenc (1986) showed that OI and the 3D-Var approach are equivalent if the cost function is defined as:

The cost function J measures:• The distance of a field x to the observations (first term)• The distance to the background xb (second term).

The distances are scaled by the observation error covariance R and by the background error covariance B respectively. The minimum of the cost function is obtained for x=xa, which is defined as the analysis.

The analysis obtained by OI and 3D-Var is the same if the weight matrix is given by

W=BHT(HBHT+R−1)−1

The difference between OI and the 3D-Var approach is in the method of solution:

• In OI, the weights W are obtained for each grid point orgrid volume, using suitable simplifications.

• In 3D-Var, the minimization of J is performed directly, allowing for additional flexibility and a simultaneous global use of the data.

The variational approach has been extended to four dimensions, by including within the cost function the distance to observations over a time interval (assimilation window). This is called four-dimensional variational assimilation (4DVar).

In the analysis cycle, the importance of the model may be stressed as:

• It transports information from data-rich to data-poor regions.

• It provides a complete estimation of the four-dimensional state of the atmosphere.

In variational assimilation, we minimize a cost function, J, which is normally a sum of two terms:

JB is the distance between the analysis and the background field

JO is the distance to the observations:

Variational Data Assimilation Formulation

A Schematic illustration of variational DA

Jb Jo

Where:x: model state variable (dimension N)xb: background fieldB: background-error covariance matrixH: forward interpolation (linear or non-

linear)y: observation vector for the observationsR: observation-error covariance matrix

In practice, the solution is obtained through minimization algorithms for J(x) using iterative methods for minimization; the conjugate gradient or quasi-Newton methods.

3D-Var minimization cost function

A schematic representation of the cost function in a simple 1-D case.   (Bouttier and Courtier, 1999)

Ensemble mean

observed (idealized!)

Ensemble members

Ensemble members

x f

x1f

xnf

time

Initial state

Uncertainties in the initial state: Ensembles

Why do we need a hybrid DA system?

• A 3D-VAR system uses only climatological (static) background error covariances.• Flow-dependent covariance through ensemble is needed.• Hybrid combines climatological and flow-dependent background error covariances.• It can be adapted to an existing 3D-VAR system.• Hybrid can be robust for small size ensembles.

Ensemble Formulation Basics

• Assume the following ensemble forecasts:

• Ensemble mean:

• Ensemble perturbations:

• Ensemble perturbations in vector form:

X f (x1f , x2

f , x3f ,.....xN

f )

xnf xn

f x f

X f (x1f ,x2

f ,x3f ,....,xN

f )

n 1,N

xnf xn

f x f

J(x1' , ) 1

1

2x1

' T B 1x1' 2

1

2 T C 1

1

2(yo ' Hx ' )T R 1(yo ' Hx ' )

x ' x1' ( k oxk

e )k1

K

3D-VAR incrementx1'

x' Total increment including hybrid

1 Weighting coefficient for static 3D-VAR covariance

2 Weighting coefficient for ensemble covariance Extended control variable

C: correlation matrix for ensemble covariance localization

1

1

1

2

1Conserving total variance requires:

The Hybrid Formulation…

Ensemble covariance is implemented into the 3D-VAR cost function via extended control variables:

WRF Ensemble:

M1

M2

M3

M4

M5

….

Mn

Compute: ensemble mean

Compute: Ensemble perturbations (ep2): u,v,t,ps,q, mean, and std_dev

WRF-VAR (VERIFY): ob.etkf ensemble

ETKF: Update ens perturbations

Update: ens. initial conditions

Update: ens. boundary conditions

WRF-VAR (QC-OBS): filtered_ob.ascii

WRF ensemble run for the next cycle

cycling

WRF-VAR (hybrid)

Update: ensemble mean

WRF deterministic forecast run

(Demirtas et al. 2009)

A hybrid (3DVAR –ETKF) system design and implementation

Data AssData Assiimmiilatlatiion Problemson Problems

Transferring information from the scattered locations and times of the observations to the model grid, while the same time.

Preserving the inter-related physical, dynamical and numerical consistency in the short-range forecast, since these are essential for making consistently good NWP forecasts.

Acknowledgements: Thanks to documents/images of ECMWF, P. Lynch (UoD) and various others that provided excellent starting point for this talk!

Thanks for attending….