materials process design and control laboratory 1 high-dimensional model representation technique...

CCOORRNNEELLLL U N I V E R S I T Y


Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory 11

High-dimensional model representation High-dimensional model representation technique for the solution of stochastic PDEs technique for the solution of stochastic PDEs

Nicholas Zabaras and Xiang MaMaterials Process Design and Control Laboratory

Sibley School of Mechanical and Aerospace Engineering101 Frank H. T. Rhodes Hall

Cornell University Ithaca, NY 14853-3801

Email: [email protected]: http://mpdc.mae.cornell.edu/

CSE09, SIAM Conference on Computational Science and Engineering, Miami, FL, March 2-6, 2009




Outline of the presentation

Problem definition

Basic concepts of high-dimensional model

representation (HDMR)

Adaptive sparse grid collocation (ASGC) method for

interpolating component functions

Examples

Conclusions




Motivation

Why uncertainty modeling ?

Assess product and process reliability

Estimate confidence level in model predictions

Identify relative sources of randomness

Provide robust design solutions

All physical systems have inherent associated randomness

SOURCES OF UNCERTAINTIES

• Multiscale nature inherently statistical

• Uncertainties in process conditions

• Material heterogeneity

• Model formulation – approximations,

assumptions




Motivation Previous developed conventional and adaptive collocation methods are not suitable for high- dimensional problems due to their weakly dependence on the dimensionality (logarithmic) in the error estimate. Although ASGC can alleviate the problem to some extent, it depends on the regularity of the problem and it is only effective when some random dimensions are more important than others.

High-dimensional model representation (HDMR) is an efficient tool to capture the model output as finite number of hierarchical correlated function-expansions in terms of the inputs starting from lower-order to higher-order. There does not exist a closed form for implementation to higher-order expansion. We try to develop a general formulation of HDMR for efficient computer implementation and to achieve higher accuracy. To represent the component functions of HDMR, either low-dimensional data table or tensor-product type Lagrange polynomial interpolation are currently used, which significantly affects its accuracy of interpolation.

In the current work, we combine the strength of the HDMR and ASGC methods to develop a stochastic dimension reduction method.

CSGC




Problem definition

Define a complete probability space . We are interested to find a stochastic function such that for P-almost everywhere (a.e.) , the following equation holds:

, ,F P

:u D

, ; , ,

, ,

L u f D

B u g D

x x x

x x x

where are the coordinates in , L is (linear/nonlinear) differential operator, and B is a boundary operator.

1 2, , , dx x xx d

In the most general case, the operators L and B as well as the driving terms f and g, can be assumed random.

In general, we require an infinite number of random variables to completely characterize a stochastic process. This poses a numerical challenge in modeling uncertainty in physical quantities that have spatio-temporal variations, hence

necessitating the need for a reduced-order representation.




Representing randomness

S

Sample space of elementary events

Real line

Random

variable

MAP

Collection of all possible outcomes

Each outcome is mapped to a

corresponding real value

Interpreting random variables as functions

A general stochastic process is a random field with variations along space and time – A

function with domain (Ω, Τ, S)

1. Interpreting random variables

2. Distribution of the random variable

e.g. inlet velocity, inlet temperature

1 0.1o

3. Correlated data

ex. presence of impurities, porosity

Usually represented with a correlation function

We specifically concentrate on this




Representing Randomness

1. Representation of random process

- Karhunen-Loève, Polynomial Chaos expansions

2. Infinite dimensions to finite dimensions

- depends on the covariance

Karhunen-Loèvè expansion

Based on the spectral decomposition of the covariance kernel of the stochastic process

Random process Mean

Set of random variables to

be found

Eigenpairs of covariance

kernel

• Need to know covariance

• Converges uniformly to any second order process

Set the number of stochastic dimensions, N

Dependence of variables

Pose the (N+d) dimensional problem




Karhunen-Loeve expansion

1

( , , ) ( , ) ( , ) ( )i ii

X x t X x t X x t Y

Stochastic process

Mean function

ON random variablesDeterministic functions

Deterministic functions ~ eigen-values, eigenvectors of the covariance function

Orthonormal random variables ~ type of stochastic process

In practice, we truncate (KL) to first N terms

1( , , ) fn( , , , , )NX x t x t Y Y




The finite-dimensional noise assumption

By using the Doob-Dynkin lemma, the solution of the problem can be described by the same set of random variables, i.e.

1, , , , Nu u Y Y x x

So the original problem can be restated as: Find the stochastic function such that

:u D

, ; , ,

, ; , ,

L u f D

B u g D

x Y x Y x

x Y x Y x

In this work, we assume that are independent random variables with probability density function . Let be the image of . Then

1

, N

i ii

Y

Y Y

1

N

i iY

i i iY

is the joint probability density of with support 1, , NY YY

1

NN

ii




High dimensional model representation (HDMR)1

Let be a real-value multivariate stochastic function: , which depends on a N-dimensional random vector . A HDMR of can be described by

f Y N 1 2, , 0,1

N

NY Y Y Y

f Y 1 1

1

01

, ,s s

s

N N

i i i is i i

f f f Y Y

Y

where the interior sum is over all sets of integers , that satisfy

s1, , si i

11 si i N .This relation means that

1 2 1 2 1 2 3 1 2 3

1 2 1 2 3

1 1

1

01

12 1

, , ,

, , , ,s s

s

N N N

i i i i i i i i i i i ii i i i i i

N

i i i i N Ni i

f f f Y f Y Y f Y Y Y

f Y Y f Y Y

Y

can be viewed as a finite hierarchical correlated function expansion in terms of the input random variables with increasing dimensions.

1N S N

The basic conjecture underlying HDMR is that the component functions arising in typical “real problems” will not likely exhibit high-order

cooperativity among the input variables such that the significant terms in the HDMR expansion are expected to satisfy the relation: for

1. O. F. Alis, H. Rabitz, General foundations of high dimensional representations, Journal of Mathematical Chemistry, 25 (1999) 127-142.




High dimensional model representation (HDMR)

1 2 1 2 1 2 3 1 2 3

1 2 1 2 3

1 1

1

01

12 1

, , ,

, , , ,s s

s

N N N

i i i i i i i i i i i ii i i i i i

N

i i i i N Ni i

f f f Y f Y Y f Y Y Y

f Y Y f Y Y

Y

In this expansion: denotes the zeroth-order effect which is a constant.

The component function gives the effect of the variable acting independently of the other input variables.

The component function describes the interactive effects of the variables and . Higher-order terms reflect the cooperative effects of increasing numbers of variables acting together to impact upon .

The last term gives any residual dependence of all the variables locked together in a cooperative way to influence the output .

Depending on the method to represent component functions, there are two types of HDMR: ANOVA-HDMR and Cut-HDMR.

i if Y iY

1 2 1 2

,i i i if Y Y

1iY

2iY

12 1, ,N Nf Y Y

0f

f Y

f Y




ANOVA - HDMR

ANOVA-HDMR is the same as the analysis of variance (ANOVA) decomposition used in statistics. Its component functions are given as

1

21 2 1 2 1 1 2 2

1 2

0

0

0 ,

,

N

N

N

i i jj i

i i i i j i i i ij i i

f f d

f Y f dY f

f Y Y f dY f Y f Y f

Y Y

Y

Y

This expansion is useful for measuring the contributions of the variance of individual component functions to the overall variance of the output.

A significant drawback of this method is the need to compute the above integrals to extract the component functions, which in general are high- dimensional integrals and thus difficult to carry out.

To circumvent this difficulty, a computationally more efficient cut-HDMR expansion which can be combined with ASGC is introduced in the next few slides.




Cut - HDMR

In this work, Cut – HDMR is used for the multivariate interpolation problem. For Cut-HDMR, we fix a reference point . For this work, we fix it at the mean of the random input. The component functions

of Cut-HDMR are given as follows:

1 2, , , NY Y YY

1 2

1 2 1 2 1 2 1 1 2 2

1 2 3

1 2 3 1 2 3 1 2 3 1 2 1 2 1 3 1 3

2 3 2 3 1 1 2 2 3 3

1

1 1 1 1 1

0

0

0

0

, ,

, , , , , ,

,

, ,s

s s s s

ii i i

i ii i i i i i i i i i

i i ii i i i i i i i i i i i i i i i i

i i i i i i i i i i

i ii i i i i i i i i

f f

f Y f Y f

f Y Y f Y Y f Y f Y f

f Y Y Y f Y Y Y f Y Y f Y Y

f Y Y f Y f Y f Y f

f Y Y f Y Y f Y

Y

1 1

1 1 1 2

0 1

,s

s s

N N

i i ii i i i i i

Y f Y f

where the notation stands for the function with all the remaining variables of the input random vector set to , e.g.

1

1,s

s

i ii if Y Y f Y

Y i

if Y stands for 1 1 1, , , , ,i i i Nf Y Y Y Y Y which is a univariate function.




Cut - HDMR

The cut-HDMR expansion up to the pth order with minimizes the following functional:

0 p N

1 1

1

2

01

,N p p

p

N

i i i i i i i ii i i i

J f f f Y f Y Y d d Y Y dY

Y Y Y

with the constraints 1 1, , 0, 1, 2, ,

p pi is s

i i i iY Y

f Y Y s p

The null property introduced above serves to assure that the functions are orthogonal with respect to the inner product induced by the measure

1 1 1 1

, , , , 0p p s s

N

i i i i j j j jf Y Y f Y Y d

Y

for at least one index differing in and , and may be the same as . This orthogonality condition implies that the cut-HDMR expansion to pth order is exact along lines, planes and hyperplanes of dimension up to p which pass through the reference point .

1, , pi i 1, , sj j sp

Y




Cut – HDMR VS Taylor expansion

Suppose defined in a unit hypercube can be expanded as a convergent Taylor series at reference point , i.e.,

f Y

Y

2

1 , 1

1

2!

N N

i i i i j ji i ji i j

f ff f Y Y Y Y Y Y

Y Y Y

Y Y

Y Y

It is shown that the first order component function is the sum of all the Taylor series terms which contain and only contain variable . Similarly, the second order component function is the sum of all the Taylor series terms which contain and only contain variables and .

i if Y

iY

,ij i jf Y Y

iY

jY

Thus, the infinite number of terms in the Taylor series are partitioned into finite groups and each group corresponds to one Cut-HDMR component functions.

Therefore, any truncated Cut-HDMR expansion gives a better approximation of than any truncated Taylor series because the later only contains a finite number of terms of Taylor series.

f Y




Truncation error of Cut - HDMR

Assume that all mixed derivatives that include not more than one differentiation with respect to each variable are piecewise continuous. These are the derivatives

1

1for 1 ; 1,2, , .s

s

si i

fi i N s N

Y Y

Denote 1 1sup /

s s

si i i i

YA f Y Y

and let1

11s

s

N N

i is p i i

A A

Then the truncation error1 up to pth order is pf Y f Y A

1. I. M. Sobol, Theorems and examples on high dimensional model representation, Reliability Engineering and System safety 79 (2003) 187-193.




A general formulation of Cut-HDMR

In its current form, each component function is defined through lower order component functions which are unknown a priori. Therefore, it is not easy for computer programming. We develop here a general form of Cut-HDMR.

For lower-order component functions, it is easy to rewrite them as

1 2 1 2

1 2 1 2 1 2 1 2

1 2 3 1 31 2

1 2 3 1 2 3 1 2 3 1 2 1 3

2 3 31 2

2 3 1 2 3

0

0

0

0

, ,

, , , , , ,

,

ii i i

i i i ii i i i i i i i

i i i i ii ii i i i i i i i i i i i i

i i ii ii i i i i

f f

f Y f Y f

f Y Y f Y Y f Y f Y f

f Y Y Y f Y Y Y f Y Y f Y Y

f Y Y f Y f Y f Y f

Y

Each component function is expressed in terms of the function output at a specific input. However, the procedure to derive these formulas becomes tedious to carry out along these lines.





Instead, since HDMR has an analogous structure to the many body expansion used in molecular physics to represent a potential energy surface, we generalize its idea to the current work and get the following formulation of a order expansion of Cut-HDMR:

1 1

1

01

, ,s s

s

P N

i i i is i i

f f f Y Y

Y

1

1 1 11

01

, 1 1 , ,j jL

s s j jLL

S SS S L i i

i i i i i iL j j

f Y Y f f Y Y

where each component function can be obtained via the Mobius inversion approach from number theory as follows:

thP

where follows the notation defined before with all the remaining variables of the input random vector set to .

1

1, ,j jL

j jL

i i

i if Y Y

Y





Now we have a general formulation of truncated Cut-HDMR to pth order:

1 1

1

01

, ,s s

s

P N

i i i is i i

f f f Y Y

Y

1

1 1 1

1

01

, 1 1 , ,j jL

s s j jL

L

S SS S L i i

i i i i i iL j j

f Y Y f f Y Y

Previously, the components functions (i.e. , ) of Cut-HDMR were typically provided numerically, at discrete values (often tensor

product) of the input variables to construct the numerical data tables. Then the value of the function for arbitrary input was determined by performing only low-dimensional interpolation over , , …

i if Y 1 2 1 2

,i i i if Y Y

i if Y 1 2 1 2

,i i i if Y Y

This method is computational expensive and not accurate since construction of each component function involves lower order inaccurate component functions.

Now, through the general formulation, the problem is transformed to several small sub problems to interpolate the function 1

1, ,j jL

j jL

i i

i if Y Y




Cut – HDMR and ASGC

1

1 1 11

01

, 1 1 , ,j jL

s s j jLL

S SS S L i i

i i i i i iL j j

f Y Y f f Y Y

Interpolated with ASGC

Now, instead of interpolating one N-dimensional function using ASGC, the problem is transformed to several at most P-dimensional

interpolation problems, where, in general, for real problems, forP N 1N

By using ASGC for the approximation, there is no need to search the numerical data tables. Interpolation is done quickly through simple weighted sum of the interpolation function and corresponding

hierarchical surplus. In addition, the mean and variance can be deduced easily.




Stochastic Collocation based framework

Function value at any point is simply

Stochastic function in 2 dimensions

Need to represent this function

Sample the function at a finite set of points

Use polynomials (Lagrange polynomials) to get a approximate representation

: Nf ( )

1 i MM iΘ Y

( ) ( )

1

( ) ( ) ( )M

i ii

i

f f a

Y Y YI

( )f ISpatial domain is approximated using a FE, FD, or FV discretization.

Stochastic domain is approximated using multidimensional interpolating functions




LET OUR BASIC 1D INTERPOLATION SCHEME BE SUMMARIZED AS

IN MULTIPLE DIMENSIONS, THIS CAN BE WRITTEN AS

Denote the one dimensional interpolation formula as

In higher dimension, a simple case is the tensor product formula

Conventional sparse grid collocation (CSGC)

Using the 1D formula, the sparse interpolant , where is the depth of sparse grid interpolation and is the number of stochastic dimensions, is given by the Smolyak algorithm as

,q NA

00,q q N

q

1i i iU U 0 0U

Here we define the hierarchical surplus as:




Choice of collocation points and nodal basis functions In the context of incorporating adaptivity, we have used the Newton-Cotes grid using equidistant support nodes and use the linear hat function as the univariate nodal basis.

ijY

12i ijY

12i ijY

1

Furthermore, by using the linear hat function as the univariate nodal basis function, one ensures a local support in contrast to the global support of Lagrange polynomials. This ensures that discontinuities in the stochastic space can be resolved. The piecewise linear basis functions can be defined as




Adaptive sparse grid collocation (ASGC)1

Let us first revisit the 1D hierarchical interpolation For smooth functions, the hierarchical surpluses tend to zero as the interpolation level increases.

On the other hand, for non-smooth function, steep gradients/finite discontinuities are indicated by the magnitude of the hierarchical surplus.

The bigger the magnitude is, the stronger the underlying discontinuity is.

Therefore, the hierarchical surplus is a natural candidate for error control and implementation of adaptivity. If the hierarchical surplus is

larger than a pre-defined value (threshold), we simply add the 2N neighbor points of the current point.

1 .X. Ma, N. Zabaras, An hierarchical adaptive sparse grid collocation method for the solution of stochastic differential equations, JCP, in press, 2009.




Integrating HDMR and ASGC

1

1 1 1

1

01

, 1 1 , ,j jL

s s j jL

L

S SS S L i i

i i i i i iL j j

f Y Y f f Y Y

1 1

1

01

, ,s s

s

P N

i i i is i i

f f f Y Y

Y

Unlike using numerical tables, there is no need to search in the table. Interpolation is done quickly through simple weighted sum of the basis functions and the corresponding hierarchical surpluses.




Integrating HDMR and ASGC

The mean of the random solution can be evaluated as follows:

Denoting we rewrite the mean as

2u To obtain the variance of the solution, we need to first obtain an approximate expression for




Implementation

The first procedure is to find the different ordered index set from the set, i.e. . 1, 2 , , 1,2, , Si i i N

The number of components is given by

0

!

( )! !

P

s

N

N s s

Although this number still grows quickly with increasing number of dimensions, we can solve each of the sub-problems more efficiently than solving the N-dimensional problem directly.

We use a recursive loop to generate all the index sets. After solving each sub problem, the surpluses for each interpolant are stored.

Each sub-problem is uniquely determined by its corresponding index set. Therefore, it is important to transform the index set to a unique key to store surplus and perform interpolation.




Implementation

We use the following convention to generate the unique key as a string

“d(dimension of subproblem)_(index set)” Only using index set is not enough. For example, for a 25 dimensional

problem, the index set 12 can be either considered as a one-dimensional problem in the 12th dimension, or as a two-dimensional

problem in the 1st and 2nd dimension. Therefore, we put a dimension in front of the index set, e.g. “1_12” and “2_12”.

To compute the value using the HDMR, it is simple to perform interpolation for different sub-problems according to the index set.

Therefore, we use <map> from the C++ standard template library to store the different interpolants. We store each interpolant according to its unique key value defined above. Then we can perform the

interpolation by calling

map[key]->interpolate()




Numerical example: Mathematical functions

It is noted that according to the right hand side of HDMR, if the function has an additive structure, the HDMR is exact, i.e. when can be

additively decomposed into functions of single variables, then the first-order expansion is enough to exactly represent the original function.

f Y

i if Y

Let us consider the following simple function:

2

1

N

ii

f Y

Y

It is expected that, no matter what the dimensionality is, it is enough to use just 1st order expansion of HDMR.

A 3rd order expansion is used although only 1st order is enough. After constructing the HDMR, we generate 100 random points in the domain and compute the normalized L2 interpolation error with the exact value. The result is shown in the following table.

Error HDMR exact

exact

y y

y





Next, we compare it using 1st HDMR with ASGC with same epsilon and CSGC.

N = 10, Level 7 CSGCN = 10, Level 7 CSGC #points#points LL2 2 ErrorError

11stst order HDMR order HDMR 12911291 2.9794046540e-052.9794046540e-05

22ndnd order HDMR order HDMR 3301633016 2.9794046540e-052.9794046540e-05

33rdrd order HDMR order HDMR 340336340336 2.9794046540e-052.9794046540e-05

Therefore, when the target function has an additive structure, the converged HDMR

is better than both ASGC and CSGC.

The #points for CSGC at q = 6 is 171425. However, to go to the next level, the #points is 652065.

As expected, 1st order HDMR is

enough to capture the function.

101

102

103

104

105

106

10-5

10-4

10-3

10-2

10-1

100

# Points

L2 e

rro

r

CSGC: q = 6

ASGC: q = 7, = 10-4

1st HDMR + CSGC : q = 7

1st HDMR + ASGC : q = 7, = 10 -4





N = 10N = 10 #points(HDMR)#points(HDMR) MaxError MaxError (HDMR)(HDMR)

#points( ASGC)#points( ASGC) MaxError MaxError ( ASGC)( ASGC)

εε = 1e-1 = 1e-1 5151 1.3886863491e-1.3886863491e-0101

221221 1.3886863491e-1.3886863491e-0101

εε = 1e-2 = 1e-2 171171 8.3629048275e-8.3629048275e-0303

14211421 8.3629048275e-8.3629048275e-0303

εε = 1e-3 = 1e-3 331331 2.1349220166e-2.1349220166e-0303

30213021 2.1349220166e-2.1349220166e-0303

εε = 1e-4 = 1e-4 12911291 1.3026665672e-1.3026665672e-0404

1262112621 1.3026656424e-1.3026656424e-0404

εε = 1e-5 = 1e-5 51315131 8.3142450165e-8.3142450165e-0606

5102151021 8.3141714420e-8.3141714420e-0606

If however, the multiplicative nature of the function is dominant then all the right hand side components of HDMR are needed to obtain the best result. However, if HDMR requires all 2N components to obtain a desired accuracy, the method

becomes very expensive. This is demonstrated through the following example.

Therefore, as expected the error threshold εε also controls the error of a also controls the error of a converged HDMR. This can be partially explained that each component has converged HDMR. This can be partially explained that each component has error error εε, and HDMR is a linear combination of components, therefore the error is , and HDMR is a linear combination of components, therefore the error is M M εε, where M is a constant., where M is a constant.

Next, we investigate the convergence with respect to the error threshold while fixing the expansion order to 1.




Numerical example: Mathematical functions Next we consider the function “product peak” from GENZ test package:

1222

1

N

i i ii

f c Y w

Y

1 2 3 4 510

-4

10-3

10-2

10-1

100

HDMR order

L2 e

rro

r

HDMR + CSGC: q = 6

Interpolation ErrorIntegration Error

The normalized interpolation error is defined the same as before. The integration error is defined as the relative error to the exact value.

So we need at least 5th order HDMR to get a converged

expansion, due to the structure of this function.

It is also interesting to note that for integration, the HDMR

converges faster than forinterpolation.

N = 10





Thus the HDMR method is indeed not optimal in this case which requires about two billion points. Therefore, HDMR is not equally useful for interpolation of all types of mathematical functions.

We will investigate further these aspects of HDMR for the solution of stochastic PDEs.

100

102

104

106

108

10-4

10-3

10-2

10-1

100

# Points

Inte

rpo

latio

n L 2

Err

or

CSGC : q = 6

ASGC : = 10 -5

5th HDMR + CSGC : q = 6

5th HDMR + ASGC : q = 6, = 10 -5

100

102

104

106

108

10-4

10-3

10-2

10-1

100

# Points

Inte

grat

ion

L 2 Err

or

CSGC : q = 6

ASGC : = 10 -5

5th

HDMR + CSGC : q = 6

5th

HDMR + ASGC : q = 6, = 10 -5





1 2 3 4 510

-3

10-2

10-1

100

101

Order

L2 in

terp

olat

ion

erro

r

Reference point 1: (0.5,,0.5)Reference point 2: (0.55,,0.55)Reference point 3: (0.1,,0.1)

1 2 3 4 510

-4

10-3

10-2

10-1

# Points

L2 in

tegr

atio

n er

ror

Reference point 1: (0.5,,0.5)Reference point 2: (0.55,,0.55)Reference point 3: (0.1,,0.1)

We consider three different reference points in the above plots. It is shown that the results near the center of the domain is more accurate. Therefore, we always take our reference point as the mean vector.




Numerical example: Stochastic elliptic problem Here, we adopt the model problem

, , , in

, 0 on

N Na u f D

u D

with the physical domain . To avoid introducing errors of any significance from the domain discretization, we take a deterministic smooth load with homogeneous boundary conditions.

2, 0,1D x x y

, , cos sinNf x y x y

The deterministic problem is solved using the finite element method with 900 bilinear quadrilateral elements. Furthermore, in order to eliminate the errors associated with a numerical K-L expansion solver and to keep the random diffusivity strictly positive, we construct the random diffusion

coefficient with 1D spatial dependence as ,Na x

1/ 2

12

log , 0.5 12

N

N n n nn

La x Y x Y

where are independent uniformly distributed random variables in the interval

, 1,nY n N 3, 3




Numerical example: Stochastic elliptic problem

In the earlier expansion,

2

1/ 2 2: exp , if 1

8n

nL

L n

and

2sin , if even

:

2cos , if odd

p

n

p

nx

nL

xn

xn

L

The parameter can be taken as and the parameter L is

pL max 1,2p cL L/cL L Lp




Numerical example: Stochastic elliptic problem This expansion is similar to a K-L expansion of a 1D random field with stationary covariance

21 21 2 2

log , 0.5 , expNc

x xa x x x

L

Small values of the correlation correspond to a slow decay, i.e. each stochastic dimension weighs almost equally. On the other hand, large

values of result in fast decay rates, i.e., the first several stochastic dimensions corresponding to large eigenvalues weigh relatively more. By using this expansion, it is assumed that we are given an analytic stochastic input. Thus, there is no truncation error. This is different from the discretization of a random filed using the K-L expansion, where for different correlation lengths we keep different terms accordingly.

In this example, we fix N and change to adjust the importance of each stochastic dimension. In this way, we investigate the effects of .

cL

cL

cL

cL

The normalized error is the same as before, where the exact solution is taken as the statistics from 106 Monte Carlo samples

2L




Numerical example: Stochastic elliptic problem N=7

1 2 3 40

0.002

0.004

0.006

0.008

0.01

0.012

Order

L2 E

rror

s

Mean : HDMR + ASGC = 10-6

Std : HDMR + ASGC = 10-6

Lc = 1/2

1 2 3 40

0.005

0.01

0.015

0.02

0.025

0.03

0.035

Order

L2 E

rror

s



Lc = 1/4

1 2 3 40

0.005

0.01

0.015

0.02

Order

L2 E

rror

s



Lc = 1/16

1 2 3 40

0.002

0.004

0.006

0.008

0.01

Order

L2 E

rror

s



Lc = 1/64

Convergence with orders of

expansion





-0.01 0 0.01 0.02 0.03 0.040

10

20

30

40

50

60

70

80

90

u

PDF

MC

HDMR, 1st

order

HDMR, 2nd

order

HDMR, 3rd

order

Lc = 1/2

-0.01 0 0.01 0.02 0.03 0.040

20

40

60

80

100

u

PDF

MC

HDMR, 1st order

HDMR, 2nd order

HDMR, 3rd order

Lc = 1/4

0 0.005 0.01 0.015 0.020

20

40

60

80

100

120

140

160

u

PDF

MC

HDMR, 1st

order

HDMR, 2nd

order

HDMR, 3rd

order

Lc = 1/16

0.005 0.01 0.0150

50

100

150

200

250

300

350

u

PDF

MC

HDMR, 1st

order

HDMR, 2nd

order

HDMR, 3rd

order

Lc = 1/64

PDF at

(0.5,0.5)





101

102

103

104

105

106

10-4

10-3

10-2

10-1

100

# Points

L2 E

rror

of

Std

CSGC : q = 7

ASGC : = 10-6

3rd HDMR + CSGC : q = 7

3rd HDMR + ASGC : = 10-6

MC = 1,000,000

Lc = 1/2

101

102

103

104

105

10-4

10-3

10-2

10-1

100

# Points

L2 e

rror

of

Std

CSGC : q = 7

ASGC : = 10-8

2nd HDMR + CSGC : q = 7

2nd HDMR + ASGC : = 10-8

MC = 1,000,000

Lc = 1/16

101

102

103

104

105

10-4

10-3

10-2

10-1

100

# Points

L2 e

rror

of

Std

CSGC : q = 7

ASGC : = 10-9

2nd

HDMR + CSGC : q = 7

2nd

HDMR + ASGC : = 10-9

MC = 1,000,000

Lc = 1/64

101

102

103

104

105

106

10-3

10-2

10-1

100

# Points

L2 e

rror

of

Std

CSGC : q = 7

ASGC : = 10-7

3rd

HDMR + CSGC : q = 7

3rd


MC = 1,000,000

Lc = 1/4





The convergence of mean is faster than that of std, since mean is a first order statistics and first order expansion is enough to give good result.

For large correlation length, in order to achieve the same accuracy as other methods, a higher order expansion is needed since the problem is

highly anisotropic and higher order effects for std cannot be neglected. However, as seen from the plots, as the correlation length decreases, the problem becomes smoother in the random space, even first order

expansion can give enough accuracy for std, e.g. when Lc = 1/64, 1st order can give a error of O(10-3).

For large correlation length, ASGC is still the most effective method. When correlation length is small, the effect of ASGC becomes the same as CSGC. In this case, lower order expansion of HDMR is enough and thus becomes favorable than others.

From the PDF figures, we can have the same conclusions. For small correlation length, the PDF of first order expansion is nearly the same as

that of MC result.





101

102

103

104

105

106

10-3

10-2

10-1

100

# Points

L2 e

rror

of

std

CSGC : q = 4

2nd

HDMR + CSGC : q = 4MC = 1,000,000

N = 21, Lc = 1/64

101

102

103

104

105

106

10-4

10-3

10-2

10-1

# Points

L2 e

rror

of

mea

n

CSGC : q = 4

2nd

HDMR + CSGC : q = 4MC = 1,000,000

N = 21, Lc = 1/64

0 0.2 0.4 0.6 0.8 10

0.002

0.004

0.006

0.008

0.01

x

Mea

n

MC = 1,000,000

1st

HDMR + CSGC : q = 4

2nd

HDMR + CSGC : q = 4

N = 21, Lc = 1/64

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1.2

1.4x 10

-3

x

Std

MC = 1,000,000

1st

HDMR + CSGC : q = 4

2nd

HDMR + CSGC : q = 4

N = 21, Lc = 1/64

Std along y = 0.5





For such a small correlation length, the effect of ASGC is the same as CSGC. So we only use HDMR + CSGC.

In such a high dimensional problem, CSGC or ASGC is not optimal in the sense that a large number of collocation points is needed to get good

accuracy. The level 4 dimension 21 CSGC has 145377 points, where the convergence rate is even worst than MC for std.

On the other hand, 2nd order HDMR + level 4 CSGC can give us a much higher accuracy for both mean and std, and the convergence rate is much better than MC method.

To improve the accuracy for CSGC method, we need to go to the next interpolation level, where the number of points is 1285761 which is definitely not good compared with MC method . Therefore, it is clear that sparse grid collocation method still depends weakly on the dimensionality

of the problem.




Numerical example: Flow through random media

Basic equation for pressure and velocity in a domain

where denotes the source/sink term.

To impose the non-negativity of the permeability, we will treat the permeability as a log- normal random field obtained from the K-L expansion

injection well

production well

1 1 f x

exp( )k Y

where is a zero mean Gaussian random field with covariance function Y

2

2, expr

Cov x yL





We first fix and vary the correlation length, where the KLE is truncated such that the eigenvalues correspond to

99.7% of the energy.

2 1

0 0.2 0.4 0.6 0.8 10.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

x

Stan

dard

dev

iati

on

MC (1,000,000)

HDMR, 1st

order

HDMR, 2nd

order

HDMR, 3rd

order

L = 1.0, N = 6

0 0.2 0.4 0.6 0.8 10.006

0.008

0.01

0.012

0.014

0.016

0.018

0.02

0.022

x

Stan

dard

dev

iati

on

MC (1,000,000)

HDMR, 1st order

HDMR, 2nd

order

L = 0.5, N = 10

0 0.2 0.4 0.6 0.8 10.016

0.018

0.02

0.022

0.024

0.026

0.028

0.03

x

Stan

dard

dev

iati

on

MC (1,000,000)

HDMR, 1st

order

L = 0.1, N = 25

Std along y = 0.5

It is noted that in all three cases, even first-order expansioncan give accuracy as good as O(10-3). Higher-order terms do not provide significant improvements.

The reason is that when the truncated KLE can accurate represent the random field, the random variables are

uncorrelated. Therefore, their cooperative effects on the output are very weak while the individual effects have strong impact on the output.





101

102

103

104

105

106

10-3

10-2

10-1

100

101

# Points

L2 e

rror

of

std

CSGC : q = 8

ASGC : = 10-4

1st

HDMR + CSGC : q = 10

1st


MC (1,000,000)

L = 1.0

101

102

103

104

105

106

10-3

10-2

10-1

100

101

102

# Points

L2 e

rror

of

std

CSGC : q = 4

ASGC : = 10-4

1st


1st


MC (1,000,000)

L = 0.1

101

102

103

104

105

106

10-3

10-2

10-1

100

101

102

#Points

L2 e

rror

of

std

CSGC : q = 6

ASGC : = 10-5

1st


1st


MC (1,000,000)

L = 0.5

Since only first-order expansion is used, the convergence of HDMR + ASGC is very obvious. The # points is nearly two-orders of magnitude lower compared with the MC method.

The employed covariance kernel has a fast decay of eigen-values. Therefore, even correlation L= 0.1 will result in

different importance in each dimension, which can be effectively detected by the ASGC method.

The MC result is obtained with 50, 100, 100 terms in KLE expansion, respectively, which also shows that the truncation error of KLE is negligible.





101

102

103

104

105

106

10-3

10-2

10-1

100

101

# Points

L2 e

rror

of

std

CSGC: q = 6

ASGC : = 10-5

1st HDMR + CSGC : q = 10

1st HDMR + ASGC : = 10-5

Lc = 0.5, 2 = 0.25

Next we fix the correlation length and vary the variance of the covariance to account for large variability in the input

uncertainty.

101

102

103

104

105

106

10-3

10-2

10-1

100

101

102

# Points

L2 e

rror

of

std

CSGC : q = 6

ASGC : = 10-4

1st


1st


MC : 1,000,000

L = 0.5, 2 = 4.0

101

102

103

104

105

106

10-3

10-2

10-1

100

101

102

# Points

L 2 err

or o

f st

d

CSGC : q = 6

ASGC : = 10-5

1st


1st


MC : 1,000,000

L = 0.5, 2 = 2.0

The coefficient of variation is 53%, 253%, 732%, respectively. Although 10 KLE terms are used for calculation, 100 terms are

kept for MC simulation .

Again, first-order expansion successfully captures the input uncertainty





0 0.2 0.4 0.6 0.8 1

0.02

0.022

0.024

0.026

0.028

0.03

x

Stan

dard

Dev

iati

on

1st HDMR + ASGC : = 10-5

MC (1,000,000)

L = 1/64, N = 150

102

103

104

105

10-3

10-2

10-1

100

101

# Points

L2 e

rror

of

std

1st

order HDMR + ASGC : = 10-5

-1

-1/2

L = 1/64, N = 150

1/ 64, 150L N

Std along y = 0.5 Error convergence

Relative error is 36.635 10




Conclusions

A general framework for stochastic high-dimensional model representation technique is developed. It applies the deterministic HDMR in the stochastic space

together with ASGC to represent the stochastic solutions.

Various statistics can be easily extracted from the final reduced representation. Numerical examples show the efficiency and accuracy for both stochastic

interpolation and integration.

Numerical examples show that for problems with not-equally weighted random dimensions, a higher-order expansion is needed to get accurate results so that ASGC remains favorable in this case. On the other hand, for small correlation length when all dimensions weigh equally, the lower-order HDMR combined with ASGC or CSGC has the best convergence rate over MC and direct CSGC method.

For high-dimensional problems with equally weighted random dimensions, the sparse grid collocation method depends strongly on the dimensionality and HDMR is the best choice.

It is interesting that when the random input was truncated from KLE, only 1st order expansion was enough to capture the input uncertainty

materials process design and control laboratory 1 high-dimensional model representation technique...

Documents