Accounting for ensemble variance inaccuracy with Hybrid Ensemble 4D-VAR

“There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don't know we don't know.”— Donald Rumsfeld, 2/12/2002, then U.S. Secretary of Defense

Hybrid provides framework for accounting for the inaccuracy of our knowledge of unknown unknowns in environmental state estimation

Craig Bishop1, Elizabeth Satterfield2, David Kuhl3, Tom Rosmond4

1Naval Research Laboratory, Monterey, CA2NRC/Naval Research Laboratory, Monterey, CA

3NRC/Naval Research Laboratory, Washington DC, CA4Science Application International Corp., Forks, WA.

Overview• How is Hybrid-Ensemble-AR different from NAVDAS-AR?• Theoretical justification for Hybrid• Performance of Hybrid at T119L42 control, T47L42

ensemble/inner-loop. Hybrid weights from brute force tuning. We will call this the standard Hybrid.

• New theory gives weights directly from archive of (ensemble-variance, innovation) pairs using new theory. (6 regions). We call this the equation Hybrid.

• Performance comparison: tuned weights versus theoretical weights.

• Summary• Discussion: Hybrid versus LETKF/EnKF/EAKF

Note: AR=Accelerated-Representer=4D-VAR-in-obs-space

How is ensemble-AR different from NAVDAS-AR?

0 0

0 0

0 _ 0 _ 0

NAVDAS-AR has

Ensemble-AR currently enables the following option for the initial covariance

1

The code al

b b T

NAVDAS AR b b T

b b bHybrid Ensemble

P P MP

MP MP M Q

P P P

1

so allows for variations in across 6 geographical regions.We can also run the

code with the TLM and adjoint turned off by using the 4D localized ensemble covariance

1

1

KT

ENSEMBLE AR i iiK

P N x x x x

1

0 _

1

1

where is ensemble perturbation, is localization matrix and is TLM of balance

operator ( is optional). is ob

T

KT T T

i ii

i

bEnsemble

K

C N

N x x x x WW N

x x C N

N P

tained by using the above equation but only computing

the part of it that pertains to the initial time.

Theoretical justification for Hybrid arises from treatment of the hidden volatility problem

Deep trough over NE US

Ensemble based uncertainty prediction

Hidden volatility: When the error variance (volatility) depends on a flow pattern that is unlikely to repeat itself, it is hidden because one or two realizations of error do not enable an estimation of variance.

Replicate Earths for ultimate uncertainty quantification

• Imagine an unimaginably large number of quasi-identical Earths.

Each Earth has one true state and one prediction but these differ from one Earth to another.

Collect all Earths having the same true state but differing forecasts of this state to

defin . T|e

t f

f t x x

x x

he error variance is the mean square difference

between individual forecasts and the single truth within this

s

true

et.

Hybrid DA and uncertainty quantification in presence of Hidden Volatility (1D idealization)

1. Assume distribution ( ) of true climatological innovation variances is

an inverse-gamma distribution. (innovation )fy H x

2

is the forecast error covariance and

, where 0 and f t f f f

t R

x x t

Note that inverse-gamma ensures that innovation variance is never equal to zero.

Quantifying uncertainty in presence of Hidden Volatility (1D idealization)

2 2 2 2

2 2min

2. Assume that distribution | of ensemble sample variances given

is a gamma distribution with mean and variance determined

by an effective ensemble size . is the observatio

s s

s s a R

M R

2

2

n error variance and is a constant.

3. Use Bayes' theorem to find pdf | of true variances given the

sample variance .

a

s

s

Quantifying uncertainty in presence of Hidden Volatility (1D idealization)

2 2min

2 2min

The equation implies a naive estimate of the true

innovation variance given by =

.

Can show that the posterior mean innovation variance has th

1 where

e form

post n prior

n

s s a R

s sR

a

w w

When ensemble variance prediction is imperfect, the "best" error variance estimate

is a linear combination

is a weight between 0 and 1.

flow dependent static of a estimate and a e

Am

s

o

ti

un

mate

ts

.

to

w

a 1st principles motivation for Hybrid.

Description of Experiment

• Cycling analysis from Nov. 20, 2008 to Dec. 31, 2008• Assimilating only conventional observations (no radiances)• Background error covariance matrix at beginning of DA time

window (Pb0) is a combination of 75% Pb0_static and 25% Pb0_ensemble

Pb0_hybrid=0.75*Pb0_static+0.25*Pb0_ensemble• 32 member ET ensemble (Bishop&Toth, 99,McLay et al 08)• Model resolution: T119L42 Outer, T47L42 Inner • Skip first 10 days of analysis for spin-up• 5-day forecasts from each analysis• Verification of forecasts with Radiosondes• Comparison with current NAVDAS-AR at same resolution using

static covariance model

Global: TLM-AR (blue) vs. pb0_a025 (red)

Contours filled with red indicate regions where Pb0_hybrid reduced rms error (as measured by radiosondes) by more than 5% relative to Pb0_static. The stronger the tone of red the greater the percentage reduction. Colored blocks indicate the significance of difference with red again signifying superiority of Pb0_hybrid.

0 _ 0 _ 0 _ 0 _

The red in the panels below show the improvement of the DA system when the

initial Hybrid error covariance model

0.25 0.75 is used instead of just

These res

b b b bHybrid Ensemble Static Static P P P P

ults are for a relatively small ensemble (32 members).

Hybrid_Ensemble_AR performed significantly better than current system

Temp: TLM-AR (blue) vs. pb0_a025 (red)

Height: TLM-AR (blue) vs. pb0_a025 (red)

Rel. Humid.: TLM-AR (blue) vs. pb0_a025 (red)

Vec-Wind: TLM-AR (blue) vs. pb0_a025 (red)

Derivation of Hybrid weights from analysis of innovations

• Assuming that all of the assumptions of simple model of error variance prediction are satisfied, Hybrid weights for optimal error variance prediction may be derived using appropriate regression formula from a large number of independent (ensemble-variance, innovation) pairs.

Comparison of variances from globally tuned Hybrid weights with those from theoretically optimal weights for 6 regions (SH, Tropics, NH all below 400 hPa and then all 3 again above 400 hPa)

Average ensemble variance Static variance from Pb0

Hybrid variance (theoretical weights) Hybrid variance (tuned weights)

U-wind 850mb

Raw 6 hr ET variance exhibits large regions of very small variance. This spot is on the West Coast where radiosondes are present but the ensemble variance is below 2.

Equation hybrid variance exhibits a smaller range of variances. The analysis will be able to draw to the observations near the West Coast.

Test of ensemble-AR using eq Hybrid

• Machine: 8 processor Linux cluster• 32 member ET ensemble• Observations: All conventional obs• Period: 11/20/08 to 12/31/08 (1st week

removed from assessment and last 5 days from assessment of 120 hr forecasts)

• 6 hr DA cycle.

Global: TLM-AR (blue) vs. pb0_206E1 (red)

Equation Hybrid

Globally, max improvements larger for equation-Hybrid than for standard-Hybrid but area of significant improvement larger for standard Hybrid

Global: TLM-AR (blue) vs. pb0_a025 (red)

Globally, max improvements larger for equation-Hybrid than for standard-Hybrid but area of significant improvement larger for standard Hybrid

Standard Hybrid

Temp: TLM-AR (blue) vs. pb0_206E1 (red)

Height: TLM-AR (blue) vs. pb0_206E1 (red)

Equation Hybrid

Regionally, the equation-Hybrid is better at avoiding large areas of degradation than the standard-Hybrid.

(tropics particularly)

Temp: TLM-AR (blue) vs. pb0_a025 (red)

Height: TLM-AR (blue) vs. pb0_a025 (red)Standard Hybrid

Regionally, the equation-Hybrid is better at avoiding large areas of degradation than the standard-Hybrid.

(tropics particularly)

Rel. Humid.: TLM-AR (blue) vs. pb0_206E1 (red)

Vec-Wind: TLM-AR (blue) vs. pb0_206E1 (red)

Equation Hybrid

Regionally, the equation-Hybrid is better at avoiding large areas of degradation than the standard-Hybrid.

(tropics particularly)

Rel. Humid.: TLM-AR (blue) vs. pb0_a025 (red)

Vec-Wind: TLM-AR (blue) vs. pb0_a025 (red)

Standard Hybrid

Regionally, the equation-Hybrid is better at avoiding large areas of degradation than the standard-Hybrid.

(tropics particularly)

Summary• All error variance predictions are inaccurate• When two or more independent error variance predictions

are available, linear combination is more accurate than either one individually – hybrid allows such combinations

• Brute force tuning of Hybrid weights is very expensive• Analytical theoretical model for error variance prediction has

been developed that allows weights to be derived directly from archive of (ensemble-variance, innovation) pairs.

• Low-resolution experiments using Navy forecast model indicate that (a) tuned global weights (equation Hybrid) make Hybrid-ensemble-AR - superior to standard-AR (b) regional weights from theory (equation Hybrid) better than tuned global weights.

Discussion: Hybrid VAR/AR versus LETKF/EnKF/EAKF

• Hybrid still works well when only, say, 2 ensemble members can be generated.

• TLM/adjoint are optional in Hybrid 4DVAR/AR but adaptive ensemble covariance localization would be required for long time windows – convenient for coupled models.

• LETKF/EnKF/EAKF ensemble covariance localization is problematic for obs that are integrals of the state.

• Balance constraints easy to apply in Hybrid, difficult in LETKF/EnKF/EAKF.

• As yet, fully consistent ensemble update is only possible using perturbed obs with Hybrid. LETKF and EAKF allow update without perturbed obs.

• Ensemble covariance localization: freedom from obs space makes adaptive, spectral and statistical localization either easier or the same.

06Z

Ensemble based localization moves about 1000 km in 12 hrs. This is >=half-width of a typical LETKF observation volume (~900km).

Example of a column of the localization with 128 s s K C C

Application to global NWP modelApplication to global NWP model

Example of a column of the localization with 128 s s K C C18Z

Naval Research Laboratory Marine Meteorology Division Monterey, CaliforniaNaval Research Laboratory Marine Meteorology Division Monterey, California

Ensemble based localization moves about 1000 km in 12 hrs. This is >=half-width of a typical LETKF observation volume (~900km).

Application to global NWP modelApplication to global NWP model

<vv> Increment 06Z

Example of a column of with 128 fK s s K P C C

Statistical TLM implied by mobile adaptively localized covariance propagates single observation increment 1000 km in 12 hrs.

Application to global NWP modelApplication to global NWP model

Example of a column of with 128 fK s s K P C C

Statistical TLM implied by mobile adaptively localized covariance propagates single observation increment 1000 km in 12 hrs.

Application to global NWP modelApplication to global NWP model

<vv> Increment 18Z

1-point Covariance (Hodyss, 2008 presentation)

Initial Time Final Time

Red – True 1-point covarianceGreen – Non-adaptively localized covarianceBlue – Multi-scale DAMES covariance