karhunen-loève decomposition of gaussian measures on...

Karhunen-Loève decomposition of Gaussian

measures on Banach spaces

Jean-Charles Croix

GT APSSE - April 2017, the 13th

joint work with Xavier Bay.

1 / 29

Sommaire

1 Preliminaries on Gaussian processes

2 Gaussian measures on Banach spacesDenitionsKarhunen-loève decomposition in the Hilbert caseKarhunen-loève decomposition on Banach spaces

3 Examples on continuous functions

4 Conclusions

2 / 29

Preliminaries on Gaussian processes

Square integrable stochastic process

Consider:

T a non-empty set,

(Ω,F ,P) a probability space.

A square integrable stochastic process is a collection X = (Xt)t∈T ofsquare integrable real random variables:

∀t ∈ T , Xt : (Ω,F ,P)→ (R,B(R)) ∈ L2(Ω)

Mean function and covariance kernel

Let X be a square integrable process, then

∀t ∈ T , m(t) = E [Xt ],

∀(t, s) ∈ T 2, K (s, t) = E [(Xt −m(t))(Xs −m(s))].

From now on, all processes will be centered: m ≡ 0.

3 / 29


The covariance kernel K is a symmetric kernel:

∀(s, t) ∈ T 2, K (s, t) = K (t, s),

and of positive type:

∀n ∈ N, ∀(t1, ..., tn) ∈ T , ∀(α1, ..., αn) ∈ Rn,

n∑i=1

αiαjK (ti , tj) ≥ 0.

In short, K is a positive denite kernel on T .

Gaussian process

A square integrable stochastic process X is Gaussian if and only if:

∀n ∈ N, ∀(t1, ..., tn) ∈ T n, (Xt1 , ...,Xtn) is a Gaussian vector.

Equivalently,

∀µ =n∑

i=1

αiδti , 〈X , µ〉 =n∑

i=1

αiXti is a real Gaussian variable.

4 / 29


Gaussian space

Let X be a Gaussian process X on T , the associated Gaussian space is

HX = Xt , t ∈ T L2(Ω,F,P)

The mapping

U : Z ∈ HX → (t → 〈Z ,Xt〉L2) ∈ RT

is a linear and injective.

Reproducing Kernel Hilbert space (RKHS)

Let X be a Gaussian process on T , the associated RKHS is

HX = U(HX ).

It is a Hilbert space with 〈f , g〉HX= 〈U−1(f ),U−1(g)〉HX

as innerproduct.

5 / 29


The previous mapping U is an isometric isomorphism, thus

HK ≡ HK as Hilbert spaces.

Moore-Aronszajn theorem

Let T a set and K a positive-denite kernel on T , then there exists aunique Hilbert subspace HK ⊂ RT such that:

1 HK = K (t, .), t ∈ T ,2 ∀t ∈ T , ∀h ∈ HK , h(t) = 〈h,Kt〉HK

.

Conversely, let T be a set and K a positive denite symmetric kernel,there exists a Gaussian process on T with K as covariance kernel(Kolmogorov extension theorem).

6 / 29


Let T be a set and CT the cylindrical σ-algebra on RT . Any stochasticprocess X on T is a (CT ,F)-measurable mapping

X : (Ω,F ,P)→ (RT , CT ).

The process X denes a probability measure on (RT , CT )

µX = P X−1.

The dual space(RT)∗

= span(δt , t ∈ T ) and

∀µ ∈(RT), µX µ−1

is a Gaussian measure on (R,B(R)).

This property will be fundamental in the rest of this talk.

7 / 29

Gaussian measures on Banach spaces

From now on:

(E , ‖.‖E ) will always be a real (separable) Banach space,

B(E ) its Borel σ-algebra.


A probability measure γ on (E ,B(E )) is (centered) Gaussian if and onlyif γ f −1 is a (centered) Gaussian measure on (R,B(R)) for all f ∈ E∗.

We will always consider centered Gaussian measure. The characteristicfunctional is well dened and

∀f ∈ E∗, γ(f ) :=

∫E

e i〈x,f 〉E,E∗γ(dx) = e−Cγ (f ,f )

2 ∈ R

with Cγ(f , g) =∫E〈x , f 〉E ,E∗〈x , g〉E ,E∗γ(dx) (covariance).

8 / 29


The covariance is bilinear, symmetric, positive and continuous.

Covariance operator

Let γ be a Gaussian measure on (E ,B(E )) then

Rγ : f ∈ E∗ →∫E

〈x , f 〉E ,E∗xγ(dx) ∈ E

is the covariance operator associated to γ.

It is characterized by the following relation

∀(f , g) ∈ E∗ × E∗, 〈Rγ f , g〉E ,E∗ = Cγ(f , g).

Moreover, it is a positive symmetric kernel: ∀(f , g) ∈ E∗ × E∗

〈Rγ f , g〉E ,E∗ = 〈Rγg , f 〉E ,E∗ ,

〈Rγ f , f 〉E ,E∗ ≥ 0

and Rγ is a compact (nuclear) operator.

9 / 29


Similarly to Gaussian processes, dene on Rγ(E∗)

〈h, g〉γ = 〈Rγ h, g〉E ,E∗

with Rγ h = h and Rγ g = g . The Cameron-Martin space is

H(γ) := Rγ f , f ∈ E∗ ⊂ E

and we have the reproducing property

∀h ∈ H(γ), ∀f ∈ E∗, 〈h, f 〉E ,E∗ = 〈h,Rγ f 〉γ .

(H(γ), < ., . >γ) is a Hilbertian subspace of E ,

Rγ(X ∗) is dense in H(γ),

Rγ may not be injective.

10 / 29


As we just saw, Rγ : E∗ → E need not be injective.

However, Rγ f = 0 implies f = 0, γ-almost everywhere.

Gaussian space

Let γ be a Gaussian measure on (E ,B(E )) and

i : f ∈ E∗ → f ∈ L2(γ),

the Gaussian space is

E∗γ = i(E∗)L2(γ)

.

For all Gaussian measures, Gaussian and Cameron-Martin spaces areisometric

E∗γ ≡ H(γ).

11 / 29

Karhunen-loève decomposition on Hilbert spaces

Consider now:

(H, 〈., .〉H) a (separable) Hilbert space,

BH∗ is weakly compact,

H = H∗ (Riesz representation theorem),

γ Gaussian measure on (H,B(H)).

Continuity

h ∈ H → 〈Rγh, h〉H is weakly sequentially continuous.

As consequence, consider h0 ∈ BH such that:

sup‖h‖H≤1

〈Rγh, h〉H = 〈Rγh0, h0〉H = λ0,

then h0 is an eigenvector of Rγ associated to the eigenvalue λ0.

12 / 29


The previous h0 may be used to split H in two orthogonal subspaces

H = Rh0 ⊕⊥ H1.

From this, dene:

P0 : h ∈ H → 〈h, h0〉Hh0,γλ0 = γ P−1

0,

γ1 = γ (I − P0)−1.

Orthogonal split of the Gaussian measure

The two measures γλ0 and γ1 on (H,B(H)) are Gaussian and

γ = γλ0 ∗ γ1,Rγλ0 = P0RγP0 = λ0〈h0, .〉Hh0,Rγ1 = (I − P0)Rγ(I − P0) = Rγ − Rγλ0

with Rh0 = H(γλ0), H1 = (Rh0)⊥ both equipped with 〈., .〉γ .

13 / 29


Next, apply the splitting method to the (conditional) Gaussian measureγ1 and so on and so forth...

Properties

(hn) is orthonormal in H,

(hn) is dense in H(γ),

Rγ =∑

n≥0 λn〈., hn〉Hhn with∑

n≥0 λn <∞.

In particular, Rγ is compact (nuclear) and

h =∑n≥0

〈h, hn〉Hhn, γ a.e.

The standard theory uses the compacity of Rγ as an input and thespectral theorem. Moreover, the class of all Gaussian covariances is equalto symmetric, positive and nuclear operators.

14 / 29

Karhunen-loève decomposition on Banach spaces

The following result is already known (Theorem 3.5.1 p.112 in XX) and iscalled "Karhunen-loève" decomposition in some texts:

Decomposition theorem

Let γ be a Gaussian measure on a locally convex space E , H(γ) theCameron-Martin space, (en) an orthonormal basis, (ξn)n a sequence ofindependent standard Gaussian random variables on a probability space(Ω,F ,P) then ∑

n≥0

ξn(ω)en (1)

converges almost surely in E , and has distribution γ.

We will now construct a "particular" basis in Rγ(E∗) which correspondsto decomposition of the covariance operator.

15 / 29


Consider now the general case:

(E , ‖.‖E ) is a Banach space,

BE∗ is weakly compact (Banach alaoglu theorem),

γ Gaussian measure on (E ,B(E )).

Variance continuity

f ∈ E∗ → 〈Rγ f , f 〉E ,E∗ is weakly sequentially continuous.

∃f0 ∈ BE∗ , supf∈BX∗

〈Rγ f , f 〉E ,E∗ = 〈Rγ f0, f0〉E ,E∗ = λ0.

Now, write Rγ f0 = λ0x0 ∈ E .

Spectral theory is not dened in this context.

16 / 29


(f0, x0) will be useful to split "orthogonally" the Banach space

E = Rx0 + X1,

and similarly, dene

P0 : x ∈ E → 〈x , f0〉E ,E∗x0,

γλ0 = γ P−10

,

γ1 = γ (I − P0)−1.

Orthogonal split of the Gaussian measure

The two measures γλ0 and γ1 on (E ,B(E )) are Gaussian and

γ = γλ0 ∗ γ1,Rγλ0 = P0RγP

∗0

= λ0〈h0, .〉Hh0,Rγ1 = (I − P0)Rγ(I − P0)∗ = Rγ − Rγλ0

with Rh0 = H(γλ0), H1 = (Rh0)⊥ both equipped with 〈., .〉γ .

17 / 29


Again, repeat the splitting method on γ1.

Properties

‖xn‖E = ‖fn‖E∗ = 1,

(xn) is dense in H(γ),

Rγ =∑

n≥0 λn〈., fn〉E ,E∗xn.

Again, we have:

x =∑n≥0

〈x , f ∗n 〉E ,E∗xn, γ a.e.

andRγ =

∑n≥0

λn〈xn, .〉E ,E∗xn.

The class of Gaussian covariance operators is still an open problem.

18 / 29

Consequences

So far, we have the following remarks:

Approximation of covariance operator

In the previous decomposition, we have:∥∥∥∥∥Rγ −n∑

k=0

λk〈xk , .〉E ,E∗xk

∥∥∥∥∥ = λn+1

where the norm is in L(X ∗,X ).

Maximum variance

supf∈BE∗

〈Rγn f , f 〉E ,E∗ = supf∈BE∗

Var(〈PnX , f 〉E ,E∗) = λn

∑n≥0 λn may not be nite,

The decomposition may not be unique.

19 / 29

Specify a Gaussian measure in practice

Given a function space E , to specify a Gaussian measure on a functionspace:

choose a covariance kernel K such that:

P [(t → Xt) ∈ E ] = 1

choose directly a Covariance operator (ex: inverse of ellipticdierential operator)

Sample path from stochastic processes have properties linked to theKernel:

Continuity (Kolmogorov continuity theorem),

Integrability,

Dierentiability...

20 / 29

Example on the Wiener measure

Now, consider the standard Brownian motion with kernel K (s, t) = s ∧ t.On H = L2([0, 1],R), the decomposition is:

hn : t →√2 sin

((n +

π

2

)t),

λn =1(

n + π2

)2On E = C([0, 1],R), it becomes:

h′n : t →√2p(1 2k

2p+1 ,2k+12p+1 − 1 2k+1

2p+1 ,2k+22p+1

),

xn : t →√2p+2h′n(t),

λn =1

2p+2, n = 2p + k , k = 0, ..., 2p − 1, p ≥ 0

which represent "hat functions" on dyadic intervals.

21 / 29

Example with dierent kernels

0.0 0.2 0.4 0.6 0.8 1.0−0.2

0.0

0.2

0.4

0.6

0.8

1.0

1.2Basis functions

x0x1x2x3x4x5x6x7x8x9x10x11

Figure: Wiener measure basis functions.

22 / 29


0 2 4 6 8 1010-2

10-1

100

101 Variances

Variances

Figure: Wiener measure basis variances.

23 / 29


0.0 0.2 0.4 0.6 0.8 1.0−1.0

−0.5

0.0

0.5

1.0

1.5Basis functions

x0x1x2x3x4x5x6x7x8

Figure: Gaussian kernel basis functions.

24 / 29


0 1 2 3 4 5 6 7 810-13

10-12

10-11

10-10

10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

10-1

100

101 Variances

Variances

Figure: Gaussian kernel basis variances.

25 / 29


0.0 0.2 0.4 0.6 0.8 1.0

−0.5

0.0

0.5

1.0

Basis functions

x0x1x2x3x4x5x6x7x8x9x10x11

Figure: Matern 3/2 kernel basis functions.

26 / 29


0 2 4 6 8 1010-4

10-3

10-2

10-1

100

101 Variances

Variances

Figure: Matern 3/2 kernel basis variances.

27 / 29

Brownian motion simulation

0.0 0.2 0.4 0.6 0.8 1.0−2.5

−2.0

−1.5

−1.0

−0.5

0.0

0.5Brownian motion simulation

Sin basis with n=1e4Sin basis with n=1e2Schauder basis with n=64

Figure: Simulation of brownian motion using dierent bases.

28 / 29

Conclusion

A new method to decompose Gaussian measures on Banach spaces.

The method can be applied to non-Gaussian square integrablemeasures,

Gaussian covariance characterization is still an open problem,

Links to optimal design in Kriging,

Links with Bayesian inverse problems,

Precise quantication of uncertainty.

29 / 29

karhunen-loève decomposition of gaussian measures on...

Documents