Introduction to Data AssimilationNCEO Data-assimilation training days

5-7 July 2010

Peter Jan van Leeuwen

Data Assimilation Research Center (DARC)

University of Reading

p.j.vanleeuwen@reading.ac.uk

Basic estimation theory

Observation: T0 = T + e0

First guess: Tm = T + em

E{e0} = 0E{em} = 0

Assume a linear best estimate: Tn = a T0 + b Tm

with Tn = T + en .

1) Gives: E{en} = E{Tn-T} = E{aT0+bTm-T} = = E{ae0+bem+ (a+b-1) T} = (a+b-1) T = 0 Hence b=1-a.

Find a and b such that 1) E{en} = 0 2) E{en

2} minimal

E{e02} = s0

2

E{em2} = sm

2

E{e0em} = 0

Basic estimation theory

2) E{en2} minimal gives:

E{en2} = E{(Tn –T)2} = E{(aT0 + bTm – T)2} =

= E{(ae0+bem)2} = a2 E{e02} + b2E{em

2} = = a2 s0

2 + (1-a)2 sm2

a = __________ and sm2

s02 + sm

2

This has to be minimal, so the derivative wrt a has to be zero:2 a s0

2 - 2(1-a) sm2 = 0, so (s0

2 + sm2)a – sm

2 = 0, hence:

b = 1-a = __________ s02

s02 + sm

2

sn2 = E{en

2} = ______________ = ________ (s0

2 + sm2)2

sm4 s0

2 + s04 sm

2 s02 sm

2

s02 + sm

2

Solution: Tn = _______ T0 + _______ Tmsm

2 s02

s02 + sm

2 s02 + sm

2

___ = ___ + ___1 1 1

sm2s0

2sn2

and

Note: sn smaller than s0 and sm !

Basic estimation theory

Best Linear Unbiased Estimate BLUE

Just least squares!!!

Can we generalize this?

• More dimensions

• Nonlinear estimates (why linear?)

• Observations that are not directly modeled

• Biases

• (Is this reference to the truth necessary?)

P(u)

u (m/s)

1.00.5

The basics: probability density functions

The model pdfp (u(x1), u(x2), T(x3), … )

u(x1)

u(x2)

T(x3)

Observations

• In situ observations:

irregular in space and time

e.g. sparse hydrographic observations,

• Satellite observations: indirect

e.g. of the sea-surface

The solution is a pdf!

Bayes theorem:

Data assimilation: general formulation

Bayes’ Theorem

Conditional pdf:

Similarly:

Combine:

Even better:

Filters and smoothers

Time

Filter: solve 3D problem sequentially

Smoother: solve 4Dproblem once

Note: I assumed a dynamical (=time-varying) model here.

Most present-day data-assimilation methods assume that the model and observation pdf are Gaussian distributed. Advantage: only first two moments, mean and covariance, needed.Disadvantage: This is not always a good approximation.

Gaussian distribution:

The Gaussian assumption I

Multivariate Gaussian for n-dimensional model vector (state or parameters or a combination):

The Gaussian assumption II

The pdf of the observations is actually (and luckily) the pdfof the observations given the model state. Hence we need the model equivalent of the observations.We can express each observation as:

in which d is an m-dimensional observation vector.The observation pdf is now given by:

(Ensemble) Kalman Filter I

Use Gaussianity in Bayes at a specific time:

Hence

Complete the squares to find again a Gaussian (only for linear H !!!):

(Ensemble) Kalman Filter III

Both lead to the Kalman filter equations, which again are just the leastsquares solutions:

Two possibilities to find the expressions for the mean and covariance:1)Completing the squares2)Writing the new solution as a linear combination of model and observations, assume unbiased and minimal trace of error covariance.

Spatial correlation of SSHand SST in the Indian Ocean

x

x

Haugen and Evensen, 2002

The error covariance:tells us how model variables co-vary

Likewise you can find the covariance of a model parameterwith any other model variable.

(Ensemble) Kalman Filter IV

Interestingly

Old solution: Tn = _______ T0 + _______ Tmsm

2 s02

sm2 + s0

2 sm2 + s0

2

Hence we find the same solution, but now generalized tomore dimensions and partial or indirect observations (H operator)!

(Ensemble) Kalman Filter V

Operational steps:1)Determine mean and covariance from the prior pdf2)Use Kalman filter equations to update the model variables3)Propagate the new forward in time using the model equations4)Propagate the new covariance in time using the linearized model equations5)Back to 2)

In the ensemble Kalman filter the mean and covariance are found from an ensemble of model states/runs.

Propagation of pdf in time:ensemble or particle methods

And parameter estimation?

Several ways: 1)Augment the state vector with the parameters (i.e. add parameters to the state vector). This makes the parameter estimates time variable. How do we find the covariances? See Friday.2) Prescribe the covariances between parameters and observed variables. 3)Find best fits using Monte-Carlo runs (lots of tries, can be expensive).

Nonlinear measurement operators H(but observations still Gaussian distributed)

1) Use the nonlinear H in the Kalman filter equations (no rigorous derivation possible).2) Augment the state vector x with H(x) (still assuming Gausian pdf for new model vector).3)Solve the nonlinear problem. For large-scale problems variational methods tend to be efficient: 3DVar. (Sometimes also used when H is linear.)

Variational methods

Recall that for Gaussian distributed model and observationsthe posterior pdf can be written as:

A variational method looks for the most probable state, which isthe maximum of this posterior pdf also called the mode.

Instead of looking for the maximum one solves for the minimumof a so-called costfunction.

The pdf can be rewritten as

in which

Find min J from variational derivative:J is costfunction or penalty function

Variational methods

Gradient descent methods

J

model variable

123 4 561’

3DVar3DVar solves for the minimum of the costfucntion, which is the maximum of the posterior pdf. Advantages: 1)When preconditioned appropriately it can handle very large dimensional systems (e.g. 10^8 variables)2)It uses an adjoint, which tells us the linear sensitivity of each model variable to all other model variables. This is useful for e.g. observation impact.Disadvantages:1)The adjoint has to be coded2)It doesn’t automatically provide an error estimate. For weakly nonlinear systems the inverse of the 2nd variational derivative, the Hessian, is equal to the covariance, but it comes at great cost.

4DVarThere is an interesting extension to this formulation to a smoother.

Time

Filter:Solve a 3D problemat each observation time.

4DVar:Solve a 3D problem at the beginning of the time window usingall observations.Note we get a new full4D solution!

4DVar: The observation pdf

The part of the costfunction related to the observations can againbe written as:

But now we have sets of observations at different times tobs, andH is a complex operator that includes the forward model.If observation errors at different times are uncorrelated we find:

4DVar: the dynamical model

The dynamical model is denoted by M:

Using the model operator twice brings us to the next time step:

And some short-hand notation:

4DVar

The total costfunction that we have to minimize now becomes:

in which the measurement operator Hi contains the forward model:

defined as

Weak-constraint 4DVarSo far, we have considered errors in observations andthe model initial condition. What about the forward model?

If the forward model has errors, i.e. if there are errorspresent in the model equations the model pdf has to beextended. This leads to extra terms in the costfunction:

Parameter estimation

Bayes:

Looks simple, but we don’t observe model parameters….

We observe model fields, so:

in which H has to be found from model integrations.

And what if the distributions are non-Gaussian?

Fully non-Gaussian data-assimilation methods exist,but none has proven to work for large-scale systems.A few do have potential among which the Particle Filter.It has been shown that the ‘curse of dimensionality’has been cured….

Summary and outlook

1 We know how to formulate the data assimilation problem using Bayses Theorem.2 We have derived the Kalman Filter and shown that it is the best linear unbiased estimator (BLUE).3 We derived 3D and 4DVar and discussed some of their properties.4 This forms the basis for what is to come in the next days!5 Notation: when we talk about the state vector and its observations notation will be changed:

becomes x, and d becomes y. Beware !!!