Pierre F.J. Lermusiaux, Oleg G. Logoutov and Patrick J. Haley Jr.Mechanical Engineering

and Ocean Science and Engineering, MIT

http://modelseas.mit.edu/

Adaptive Data Assimilation and Multi-Model Fusion

We thank:Allan R. RobinsonWayne G. LeslieAOSN-II and MB06 teamsONR

Most DA schemes used for realistic studies approximate fundamental principlesThese DA schemes involve parameters, options and heuristic algorithms whose specifics impact results

In the Error Subspace Statistical Estimation system, specifics • Vary with each application and with users inputs• Are adapted with time, as a function of the available data, regional

dynamics or other considerations

Data assimilation is said to be adaptive when “parameters, functional or schemes used for DA are a (quantitative) function of the measurements: the DA learns from data”

• Here, we review and illustrate several of the ESSE adaptable components (see Lermusiaux, Physica D, 2007)

• Other Adaptive DA references (Blanchet et al, 1997; Menemenlis and Chechelnistky, 2000; etc)

Adaptive Data Assimilation

1) Adaptive Error Covariance Estimation:Adaptive Learning of the Dominant Errors in ESSE

Real-time 1996 exampleSee Lermusiaux, DAO (1999)

2) “Adaptive” Error Scaling in ESSE

Dominant Error covariance estimate can be scaled by a block diagonal matrix Γ

TB = ΓEΠE Γ

Presently, scaling is tuned by trial and error (shooting)• At t0, values in Γ are usually set within 0.3 to 0.7 (when EΠET is

set to variability, e.g. Lermusiaux, Anderson and Lozano, 2000).

• At t0 and tk’s, the tuning of Γ is done by batch. Successive batches are compared to data-model misfits and the initial error increased/decreased accordingly

• For mesoscale coastal ocean, Γ stabilizes to I after 2-7 DA cycles (days to a week)

• Each block corresponds to a state variable and is defined by one scaling factor (in atmos., Γ set to a scalar, cov. inflation)

• Scaling used in the error initialization

3) “Adaptive” Parameterization of the Truncated Errors

Uncertainties not represented by the error subspace modeled by random noise j

kn

For each state variable v, random noise is sum of additive and multiplicative noise:

: White noise reddened by Shapiro Filter

: Non-dimensional white noise factor of small amplitude (1 to 5%)

( )j j j jk k k k v

α ε= +n x

( )jk v

α

( )jk vε

Presently, scaling (increase/decrease) of these parameters by trial and error (shooting) on future data-model misfits

4) Adaptive Ensemble Size, Error Subspace Rank and Stochastic Forcing

dd

• Size of ensemble controlled in real-time by quantitative criteria• Error Subspace rank selected based on Sing. Val.• Stochastic forcing parameters should be function of data-model misfits

Ens. Size of 500Sensitivity of T. error correlation estimates to error subspace rank

Ens. Size of 500, Subspace of Rank 300

Ens. Size of 500, Subspace of Rank 100 Ens. Size of 500, Subspace of Rank 20

Sensitivity of T. err. cor. estimates to ensemble size and subspace rankEns. Size of 100 Ens. Size of 100, Subspace of Rank 20

Ens. Size of 500, Subspace of Rank 100 Ens. Size of 500, Subspace of Rank 20

Stochastic Primitive EquationModel

See Lermusiaux, JCP-2006

are here

The diagonal of time-decorrelations:

The diagonal of noise variances are chosenfunction of z only, of amplitude set to:“ε * geostrophy”

Sensitivity of T. err. cor. estimates to stochastic forcing (and error subspace rank)

Ens. Size of 500 Ens. Size of 500, Subspace of Rank 100

Ens. Size of 500, with stochastic forcing Ens. Size of 500, stochas. frc., Rank 100

Percentage of Variance Explained

• Red dash-dotted: without stochastic forcing• Blue: with stochastic forcing

Normalized, Cumulative Error Variance

5) Effect of (Adaptive) Schur product of ESSE covariances with a matrix whose values decay with distances

T and S profiles

assimilated on Aug 28,

2003

ESSE error standard deviation prediction

for surface T

Reduction (prior – posterior) of error standard deviation due to DA of T and S,

no tapering by Schur product Error reduction, with tapering by Schur product

Error Reduction no tapering by Schur product Error reduction, with tapering by Schur product

DA increments (surface T), No tapering DA increments (surface T), With tapering

Multi-Model Fusion for Ocean Predictionbased on Adaptive Uncertainty Estimation

A Methodology for Multi-Model Forecast FusionAdaptive Uncertainty Estimation Schemes• Bias Correction followed by Error Variance Estimation

Capable of operating with observational data that are limited and sparse (in space and in time) with respect to the dominant ocean scalesAdaptive/sequential, using the small samples of error estimation events (possibly 1 event)

Sequential/Adaptive SchemesUncertainty Estimation from Incomplete Data-Model Misfits1) Linear sequential bias estimator (of mimimum error variance)2) Error Variance estimator of minimum mean square errorMulti-Model Fusion based on Minimum Error Variance

Approach And AssumptionsErrors = systematic + random componentsMarkovian behavior (past errors are at least partially relevant to future errors)1) Estimate and correct the biases of each model2) Estimate the error (co)-variances of each bias-corrected model3) Optimally combine the states or forecasts

Bayesian Multi-Model Fusion

Bayesian Multi-Model Fusion (at any fixed time)

• Seek multi-model (central) forecast as a linear combination of the individual forecasts, with spatially varying weights

• Weight matrices D found to ensure that central forecast has minimum error variance and is unbiased

Logutov, O. G. Multi-Model Fusion and Uncertainty Estimation for Ocean Prediction. Ph.D. dissertation, Harvard University, 2007

(Gorokhov and Stoica, IEEE Trans. Sig. Proc., 2000)

(Interpolated on same grid)

(For uncorrelated models errors and to ensure unbiased estimator)

Employing Bayesian Multi-Model Fusion for Integration of Multiple Models Into a Single Ocean Prediction System

Consists of combining the individual forecasts based on their relative error variances (defined here as uncertainty)

• minimizes error variance of multi-model (central) forecast

• spatially varying diagonal weights have clear interpretation of Bayes factors associated with the individual models

Example: 24 hour HOPS/ROMS SST forecast, valid Aug 28, 2003

Optimal Multi-Model Fusion

We need:

Unbiased forecasts

Forecast error variances(main diagonal of )B(i)k

t t

Characteristics of Coastal Ocean Data Assimilation and Prediction:

Observational data are sparse in space and in timeData are collected at different locations for different validation eventsVolume of data changes with time

−3

−2

−1

0

1

2

3 Data−Model Misfits [oC] z = 10 m, 8/7/2003

20’ 123oW 40’ 20’ 122oW 40’

36oN

20’

40’

37oN

20’

−3

−2

−1

0

1

2

3 Data−Model Misfits [oC] z = 10 m, 8/11/2003

20’ 123oW 40’ 20’ 122oW 40’

36oN

20’

40’

37oN

20’

−3

−2

−1

0

1

2

3 Data−Model Misfits [oC] z = 10 m, 8/24/2003

20’ 123oW 40’ 20’ 122oW 40’

36oN

20’

40’

37oN

20’

Sequence of validation events => Sequential/Adaptive schemes

Sequential/Adaptive Bias Estimation

• Weights w are chosen to minimize the error variance of the bias estimate

• Model-data misfits consist of the bias and of the random forecast and observational errors

• Practical Bias Model: sequential, “level” averaged, linear misfit update

Using the above misfit definition, the error variance of the bias is:

Define:

with the unbiased estimator constraint:

(m data pts)

• Optimal Bias Model is:

• Solution to this constrained minimum error variance minimization is (see Gorokhov and Stoica, IEEE Trans. Sig. Proc., 2000):

with w =

Sequential/Adaptive Bias Estimation (continued)

Example of bias estimation for MREA 2003 Exercises (Ligurian Sea)

Bias correction from one validating event

Bias corrected 24-hour forecast profiles

Bias correction from three validating events

Bias estimation for AOSN-2Z=10 m

HOPS

ROMS

Z=150 m

Error (co)-variance EstimationGiven q realizations of (random) forecast errorthe unconstrained Maximum-Likelihood error covariance estimate

has a Wishart distribution of order qThis classic estimator has a large variance for q small

Mean-Squared Error (MSE) of any estimator of B

Look for error (co)-variance estimate as a linear combination of the classic (ML) unconstrained estimate and of a spatially constrained estimate

MSE of can be expressed via expectation and variance of quadratic forms in normal variables

Given consider a quadratic form in x

Expectation and variance of y are given by:

Error (co)-variance Estimator

• Unconstrained estimate is asymptotically unbiased but can have larger estimation error variance

• Constrained (e.g. constant on fixed depth/density levels) estimate has bias coming from structural assumption, but smaller estimation error variance

Look for uncertainty estimate in the form

Optimal lambdas are found from

where

are simple expressions in terms of trace of

Example of Uncertainty Estimate for AOSN-2

From three validation events

Presently: Error variance Estimator

Example of Central Forecast for AOSN-2

Central Forecast

HOPS ROMS

24-hour T forecast for Aug 14, 2003

RMSE of HOPS, ROMS, and Two-model (Central) 24-hour Temperature Forecasts

Z=10 m Z=150 m

By analyzing expectation and variance of quadratic forms we can compute error variance of uncertainty estimates generated from data-model misfits

(uncertainty of uncertainty)

Combine uncertainty estimation from data-model misfits

with the adaptive ensemble-based ESSE uncertainty modeling

Even though much more research on Adaptive DA is needed, results indicate that error estimates, ensemble sizes, error subspace ranks, covariance tapering parameters and stochastic error models can/should be calibrated by quantitative adaptation to observational data

New Bayesian-based fusion of multiple model estimates based on• Estimation of uncertainties (Bias + Variance) of ocean models based

on the comparison of past model estimates to measurements

• Subsequent sum of model estimates with optimum error variance asedweights

Much work remains, including• Combinations of Adaptive DA and multi-model fusion schemes

• Infer improvements needed in models (adaptive modeling)

CONCLUSIONS

Conclusions• The formalisms of Bayesian multi-model fusion, sequential bias estimation,

and forecast uncertainty estimation, suited for ocean prediction, provide the methodology for integrating multiple ocean models into a single ocean prediction system

• Multi-model fusion consists of combining the individual forecasts based on their relative uncertainties

• minimizes error variance of central forecast• spatially varying weights have clear interpretation of Bayes factors associated with the individual

models

• Sequential bias estimation is different from Dee et al. type of algorithms since we explicitly compute the error variance of bias estimate and use that error variance once new data become available. Adaptiveness of the algorithm is controlled through the prior estimate error variance which determines the effect of the prior data on the current bias estimate

• Uncertainty estimation: by analyzing expectation and variance of quadratic forms we can compute error variance of uncertainty estimates generated from data-model misfits. Therefore, estimates can be combined with model-propagated uncertainties using Bayesian principle

Error Subspace Statistical Estimation (ESSE)

• Uncertainty forecasts (with dynamic error subspace, error learning)• Ensemble-based (with nonlinear and stochastic primitive eq. model (HOPS)• Multivariate, non-homogeneous and non-isotropic Data Assimilation (DA)• Consistent DA and adaptive sampling schemes• Software: not tied to any model, but specifics currently tailored to HOPS

STOCHASTIC FORCING MODEL:Sub-grid-scales

Example of bias estimation for MREA 2003 Exercises

Bias estimation from a single validation profile

24-hour forecast profiles in the 2nd half of experiment with bias model trained

on the 1st half

Shallow water equations in the frequency domain

New HU code implemented in Matlab

whereopen boundary forcing:

Inverse solution found as:

where

Reference: Egbert G.D. and S. Erofeeva (2002). Efficient Inverse Modeling of Barotropic Ocean Tides. J.Atm.Oc.Tech., Vol. 19, pp. 183-204.

Dynamic error covarianceAdjoint of dynamics

Observational error covariance

Tidal Inversion