Computer Model Validation via Dynamic LinearModel

Fei Liu1 Liang Zhang1 Mike West1

1Institute of Statistics and Decision SciencesDuke University

Kickoff Workshop, SAMSI, September 10-14, 2006

Fei Liu, Liang Zhang, Mike West CMV-DLM

Data

0 500 1000 1500 2000 2500 3000 3500

u = 0.5

u = 0.25

u = 0.35

u = 0.45

u = 0.55

u = 0.65

u = 0.75

Computer Model Ouptuts Data

Fei Liu, Liang Zhang, Mike West CMV-DLM

Statistical Modeling of theSpatially Correlated Computer Outputs

Xt(u1)Xt(u2)

. . .Xt(un)

=

Xt−1(u1) Xt−2(u1) . . . Xt−p(u1)Xt−1(u2) Xt−2(u2) . . . Xt−p(u2)

......

. . ....

Xt−1(un) Xt−2(un) . . . Xt−p(un)

φt,1φt,2

...φt,p

+

εt(u1)εt(u2)

...εt(un)

Random walk for the TVAR coefficients,

Φt =

φt ,1φt ,2

...φt ,p

; Φt = Φt−1 + wt

Fei Liu, Liang Zhang, Mike West CMV-DLM

Gaussian Stochastic Process For the TVARinnovations

Gaussian Random Field for εt :

εt(·) ∼ GP(0, vtc(·, ·))

For finite observations specifically,εt(u1)εt(u2)

...εt(un)

∼ MVN(0, Vt × Σ(u1, . . . , un))

Power Exponential Family of spatial correlation:

c(u, u′) = exp(−β | u − u

′ |),Σi,j = c(ui , uj)

Fei Liu, Liang Zhang, Mike West CMV-DLM

Discounting Variances

Discount factor δ1 for vt to allow its stochastic changes,

v−1t | Dt−1 ∼ G(δ1nt−1/2, δ1dt−1/2)

choose n0 = 1, d0 = var(X ).δ2 for Ct ,

wt ∼ MVN(mt , Ct)

Ct | Dt−1 = (1 − δ2)Ct−1/δ2

Ct−1 = Cov(Φt−1 | Dt−1)

choose m0 = 0, C0 = 10Ip×p

Fei Liu, Liang Zhang, Mike West CMV-DLM

DLM Representation

(F , G, V , W )t = (Ft , Gt , Vt , Wt)

F′t =

Xt−1(u1) Xt−2(u1) . . . Xt−p(u1)Xt−1(u2) Xt−2(u2) . . . Xt−p(u2)

......

. . ....

Xt−1(un) Xt−2(un) . . . Xt−p(un)

Gt = Ip×p

Vt = vtΣ(u1, . . . , un)

Finally, vt and Wt are sequentially specified.

Fei Liu, Liang Zhang, Mike West CMV-DLM

Backward Sampling

Gibbs Sampler

Interested in: ({v1, . . . vT}; {Φ1, . . . ,ΦT}; {β} | DT )

Sample (β | DT , v1:T ,Φ1:T ).

Sample (v1:T ,Φ1:T | DT , β).

Sample (v1:T | DT , β).

Sample (Φ1:T | v1:T , DT , β).

Fei Liu, Liang Zhang, Mike West CMV-DLM

Sample (Vt , t = 1, . . . , T | DT , β):

(a.) Forward filtering with unknown variances.(b.) Sample (V−1

T | DT , β) ∼ G(nT /2, dT /2).

(c.) Recursively sample vt , t = T − 1, . . . , 1 from,

v−1t = δ1v−1

t+1 + G ((1 − δ1)nt/2, dt/2)

Sample (Φ1:T | DT , v1:T , β):

(a.) Forward filtering again with known v1:T .

(b.) Sample (ΦT | DT , v1:T ) ∼ MVN(mT , CT ).

(c.) Recursively sample Φt , t = T − 1, . . . , 1 from,

(Φt | DT ,Φt+1, V1:T ) ∼ MVN ((1 − δ2)mt + δ2Φt+1, (1 − δ2)Ct)

Fei Liu, Liang Zhang, Mike West CMV-DLM

Posterior Distribution of β

0 200 400 600 800 1000 1200 1400 1600 1800 20001.5

1.55

1.6

1.65

1.7

1.75

1.8

1.85Trace Plot of the MCMC Samples −− beta

0 10 20 30 40 50 60 70 80 90 100−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Lag

Auto

corre

latio

n

ACF of MCMC Samples −− beta

1.5 1.55 1.6 1.65 1.7 1.75 1.8 1.850

100

200

300

400

500

600Histogram of the Posterior Samples −− beta

0 1 2 3 4 5 6 7 8 9 100

0.5

1

1.5

2

2.5

3

3.5

4

4.5Prior Density of beta

Fei Liu, Liang Zhang, Mike West CMV-DLM

Posterior Distribution of vt

0 500 1000 1500 2000 2500 3000 35007.5

8

8.5

9

9.5

10

10.5

11

11.5

12

12.5Posterior Mean of the Standard Deviation

Fei Liu, Liang Zhang, Mike West CMV-DLM

Posterior Distribution of φ

0 500 1000 1500 2000 2500 3000 3500−3

−2

−1

0

1

2

3Posterior Mean of the AR Coefficients

Fei Liu, Liang Zhang, Mike West CMV-DLM

Spatial Interpolation —-Predict Output of a computer model with new input

At new input u, we can predict (approximate) the computermodel output its from the posterior draws.

(xt(u) | xt−1:t−p(u), Data,Φ1:T v1:T , β

)∼ N(µt(u), σ2

t (u))

µt(u) =∑

j

xt−j(u)φt ,j + ρt(u, u1:n)Σ−1(u1:n, β)

εt(u1)εt(u2)

...εt(un)

σ2

t (u) = Vt(1 − ρt(u, u1:n)Σ−1(u1:n, β)ρ(u, u1:n))

Fei Liu, Liang Zhang, Mike West CMV-DLM

Predictive Curve with Posterior Quantiles

1100 1120 1140 1160 1180 1200 1220 1240 1260 1280 1300−250

−200

−150

−100

−50

0

50

100

150

2001100 −− 1300 Sectional of Data with Confidence Bands

Posterior Predictive CurveTrue Data90% Confidence Bands90% Confidence Bands

2700 2720 2740 2760 2780 2800 2820 2840 2860 2880 2900−250

−200

−150

−100

−50

0

50

100

1502700 −− 2900 Sectional of Data with Confidence Bands

Fei Liu, Liang Zhang, Mike West CMV-DLM

Decomposition – Posterior Mean

0 500 1000 1500 2000 2500 3000−1

0

1

2

3

4

Data

Posterior Mean

Time

comp

comp

comp

comp

Decomposition Of the Posterior Mean

Fei Liu, Liang Zhang, Mike West CMV-DLM

Wave Lengths – Posterior Mean

0 500 1000 1500 2000 2500 30000

20

40

60

80

100

120Wavelength of the top 5 components

Fei Liu, Liang Zhang, Mike West CMV-DLM

Moduli – Posterior Mean

0 500 1000 1500 2000 2500 30000

0.2

0.4

0.6

0.8

1

1.2

1.4Moduli of the first 5 components

Fei Liu, Liang Zhang, Mike West CMV-DLM