the high-order stochastic sequential simulation framework · outline •limits of two-point...

The High-order Stochastic Sequential

Simulation Framework:

A review with examples

Roussos Dimitrakopoulos

COSMO Stochastic Mine Planning Laboratory - http://cosmo.mcgill.ca/

Outline

• Limits of two-point statistical methods

• Spatial statistics: Definitions of cumulants, examples and interpretations

• Estimation of non-Gaussian conditional distributions with spatial cumulants

• High-order sequential simulation

• Conclusions

Limits of two-point statistics

• Second and higher order models

• Spatial cumulants

Limits of variograms

1

Variograms EW Variograms NS

2 3

0.4

0.8

1.2

10 20 30 400

3

12

lags

(h

)0.4

0.8

1.2

10 20 30 400

1

2

3

lags

(h)

Very different

patterns

same variogram

may share the

Widely different patterns, yet same statistics up to order 2Source:

SCRAF/Journel

Image 1

Image 2

Different third- and fourth-order cumulants

High-order spatial statistics

• Variogram

• two-point variances

• function of

lags & direction

• High-order

• considers joint

neighbourhoods

of n points

? h

?h1

h2

Second and high-order geostatistics

Multiple-point geostatistics (MPS)

A training image (TI)

is the model

Algorithms:

SNESIM

FILTERSIM

SIMPAT

……

Second and high-order geostatistics

Second and high-order geostatistics

• Multiple-point geostatistics (MPS)

• What if a lot of data and NOT relate to the TI?

• Applications with relatively ‘rich’ data sets?

An example:

Spatial statistics: Definitions of

moments and cumulants

• Concepts

• Definitions

• Spatial templates

• For a non-Gaussian process, cumulants provide a measure of non-Gaussianity

• Cumulants are invariant to additive constants

• For linear process, cumulants may be expressed as higher-order correlations

• Cumulants well define mathematical objects (e.g. covariance, variogram)

High-order spatial cumulants

First-order cumulant(the mean)

The first order cumulant for 3D ergodic stationary

random function (RF) Z(x) is:

where m is the mean of Z(x)

E Z(x) m Cum[Z(x)]

Second-order cumulant(the covariance)

The second order cumulant of a 3D ergodic stationary

RF Z(x) is given by:

For zero mean RF Z(x) the second order cumulant is

the centered covariance:

22( ) ( ) Cum[ ( ), ( )] ( ) zE Z x Z x h m Z x Z x h c h

2

1( ) ( ) ( ) ( ) ( ) Cum[ ( ), ( )] ( ) z

x x

E Z x Z x h C h Z x Z x h Z x Z x h c hN

h

Z(x): a 3D ergodic stationary random function (RF)

Third- to fifth-order spatial cumulants

The third-order cumulant

3 1 2 1 2

1 2

1 3

32 3

( , ) ( ) ( ) ( )

( ) ( ) ( )

( ) ( ) ( )

( ) ( ) ( ) 2 ( )

zc h h E Z x Z x h Z x h

E Z x E Z x h Z x h

E Z x E Z x h Z x h

E Z x E Z x h Z x h E Z x

For zero mean RF

c3

z (h1,h2) E Z(x)Z(x h1)Z(x h2) h1

h2

h3

The fourth-order cumulant (zero-mean RF)

4 1 2 3 1 2 3

2 1 2 2 3 2 2 2 3 1

2 3 2 1 2

( , , ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( )

( ) ( )

z

z z z z

z z

c h h h E Z x Z x h Z x h Z x h

c h c h h c h c h h

c h c h h

Third- to fifth-order spatial cumulants

h

1h

2

h

3

The fifth-order cumulant (zero-mean RF)

5 1 2 3 4 1 2 3 4

2 1 3 3 2 4 2 2 2 3 3 1 4 1

2 3 3 2 1 4 1 2 4 3 2 1 3

( , , , ) { ( ) ( ) ( ) ( ) ( )}

- ( ) ( - , - ) - ( ) ( - , - )

- ( ) ( - , - ) - ( ) ( - , -

z

z z z z

z z z z

c h h h h E Z x Z x h Z x h Z x h Z x h

c h c h h h h c h c h h h h

c h c h h h h c h c h h h 1

2 2 1 3 3 4 2 3 1 3 2 4

2 4 1 3 2 3 2 3 2 3 1 4

2 4 2 3 1 3 2

)

- ( - ) ( , ) - ( - ) ( , )

- ( - ) ( , ) - ( - ) ( , )

- ( - ) ( , ) - (

z z z z

z z z z

z z z

h

c h h c h h c h h c h h

c h h c h h c h h c h h

c h h c h h c h4 3 3 1 2- ) ( , )zh c h h

Third- to fifth-order spatial cumulants

The third order cumulant

Experimental expression

1 2

1 2

1 2

,1 2

1 2

,

,,

1 2

;

,h h

h h

h hh h

h h

N

C h h3

3

3

T

a. and are two lags in two specific directions;

b. T is the set of replicates;

c. is the cardinality of T

d. is the cumulant at the node ( ).

,1 2,1 23

1 2

1 2

1 2, 1

,1 2 3

1( ) ( ) ( ),

{ ; ; } ,

h h

h hN

k k kh h k

h hk k k

C Z x Z x h Z x hN

x x h x h

Z(x0 )

Z(x0 h2)

0 1( )Z x h

Z(x1 h2)

Z(x1) 1 1( )Z x h

c3

z (h1,h2) E Z(x)Z(x h1)Z(x h2)

Calculation of spatial cumulants

The fourth-order cumulant

4 1 2 3 1 2 3

2 1 2 2 3 2 2 2 3 1

2 3 2 1 2

( , , ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( )

( ) ( )

z

z z z z

z z

c h h h E Z x Z x h Z x h Z x h

c h c h h c h c h h

c h c h h

1 3( )Z x h

Z(x1)

1 1( )Z x h

Z(x1 h2)

Calculation of spatial cumulants

Spatial templates

z(x) z(x1)

z(x2)

h1

h2

z(x)

z(x1)

z(x2)

h1

h2

z(x)

z(x1)

z(x2)

h2

h1

z(x) z(x1)

z(x2)

h1

h2

z(x)

z(x1)

z(x2)

h1

h2

+45o

Third-order templates (2D)

X

Y

Examples and interpretations

• Binary images

• A diamond pipe

75 m75 m

Example 1: 2D binary image

Original image Third-order cumulant

h1

h2

75 m75 m75 m

3 1 2 1 2

1 2

1 3

32 3

( , ) ( ) ( ) ( )

( ) ( ) ( )

( ) ( ) ( )

( ) ( ) ( ) 2 ( )

zc h h E Z x Z x h Z x h

E Z x E Z x h Z x h

E Z x E Z x h Z x h

E Z x E Z x h Z x h E Z x

Third-order

Original image Third-order cumulant

h1

h2

125 m125 m

125 m125 m

Example 1: 2D binary image

125 m

Original image fifth-order cumulant

5 1 2 3 4 1 2 3 4

2 1 3 3 2 4 2 2 2 3 3 1 4 1

2 3 3 2 1 4 1 2 4 3 2 1 3

( , , , ) { ( ) ( ) ( ) ( ) ( )}

- ( ) ( - , - ) - ( ) ( - , - )

- ( ) ( - , - ) - ( ) ( - , -

z

z z z z

z z z z

c h h h h E Z x Z x h Z x h Z x h Z x h

c h c h h h h c h c h h h h

c h c h h h h c h c h h h

1

2 2 1 3 3 4 2 3 1 3 2 4

2 4 1 3 2 3 2 3 2 3 1 4

2 4 2 3 1 3 2

)

- ( - ) ( , ) - ( - ) ( , )

- ( - ) ( , ) - ( - ) ( , )

- ( - ) ( , ) - (

z z z z

z z z z

z z z

h

c h h c h h c h h c h h

c h h c h h c h h c h h

c h h c h h c h4 3 3 1 2- ) ( , )zh c h h

Fifth-order

Original image

125 m125 m h1

h2

h4

h3

5 { , } vector space 5 1 2

1 2

2 3 2 1 1 3 2 1 1 3 1 2

2

( , ,0,0)

{ ( ) ( ) ( ) ( ) ( )}

- (0)( ( - , - ) ( - , - ) ( , ))

-

z zx y

z z z z

c c h h

E Z x Z x Z x Z x h Z x h

c c h h h c h h h c h h

c

1 3 2 2 3 2

2 2 3 1 1 3 1

2 2 1 3

( )( ( , ) 2 ( ,0))

- ( )( ( , ) 2 ( ,0))

- ( - ) (0,0)

z z z

z z z

z z

h c h h c h

c h c h h c h

c h h c

Example 1: 2D binary image

Example 2: 2D continuous image

Original image Third-order cumulant

78 000 nodes

10 000 nodes

h1

h2

(Srivastava and Isaaks, 1989)

460 nodes

Example 3: 3D image - diamond pipe

(Fox diamond pipe, Ekati mine, NWT, Canada)

(Nowicki et al., 2004)

Example 3: 3D image - diamond pipe

(third-order)

x

y

z

x

y

y

z

x

z

Example 3: 3D image - diamond pipe

2D Cross-sections

Fourth-order Fifth-order

3rd-Order 4th-Order 5th-Order

z

x

y

z

x

y

Example3: Drill hole data, diamond pipe

A high-order sequential

simulation algorithm

• Estimating conditional distributions

• Examples

0 }n = {d

1 0( ; | )Zf z u( )

1 1( ) ( )lZ zu u

( )1 1, ( )}l

n z = {d u

2 1( ; | )Zf z u( )

2 2( ) ( )lZ zu u

...

( ) ( )1, ( ),..., ( )}l l

N n Nz z {d u u

1( ; | )Z N N Nf z u( )( ) ( )l

N NZ zu u

High-order sequential simulation

Sequential simulation

• Multivariate Legendre series

• The conditional density of Z0 given Z1=a1,…,Zn=an is

given by

= g(ci0i1…in) and ci1i2…in = cum(Xi00,X

i11,…,X

inn)NN iiiL ,,..., 10

),(...)(

1

),...,,()(

1)/(

0,,...,,

0000

10

0

00

0110

10

0

zPLdzf

zzzfdzf

zf

iiiii

iiiD

n

D

Z

NN

NN

x

x

Z

Z

Z

Legendre

cumulantsLegendre

polynomials

Order of the

approximation

High-order sequential simulation

• Multivariate Legendre-like orthogonal splines

• The conditional density of Z0 given Z1=a1,…,Zn=an is

given by

= g(ci0i1…in) and ci1i2…in = cum(Xi00,X

i11,…,X

inn)NN iiiL ,,..., 10

Legendre

cumulants

Legendre-like

orthogonal

splines

Order of the

approximation

High-order sequential simulation

)..(...)(

1

),...,,()(

1)/(

0,,...,,

0000

10

0

00

0110

10

0

zSLdzf

zzzfdzf

zf

iiiii

iiiD

n

D

Z

NN

NN

x

x

Z

Z

Z

?i0

i1

i2i3

i4

i5

i0=i1=…=in=1

Multiple point statistics

(Journel, 1990’s, and on)

Estimating conditional distributions

),(...)(

1

),...,,()(

1)/(

0,,...,,

0000

10

0

00

0110

10

0

zPLdzf

zzzfdzf

zf

iiiii

iiiD

n

D

Z

NN

NN

x

x

Z

Z

Z

Calculating cumulants when simulating

h3

u0+h3

h2

u0+h2

u0+h1h1

u0

Node to Simulate 1, order = 6,

calculate cumulants from data

Node to Simulate 2, order = 6,

calculate up to order 4 from data,

and the rest from a Training

Image

h4

h5

u0+h4

u0+h5

h2

u0+h2

h3

u0+h3

u0+h1h1

u0

Using the first the cumulants

c1, c2 and c3 of the true

distribution.

Estimating conditional distributions

Using cumulants up to

orders 12 and 25

Mixture of Gaussians

),( yxf

).()(),( ,

12

0

12

0

12 yPxPLyxf nmnm

nm

).()(),( ,

0

12

0

12 yPxPLyxf nmnm

m

nm

L1,1 . . L1,12

. .

. .

L12,1 . . L12,12

L1,1

.

.

L12,1 . . L12,12

Bivariate lognormal

Estimating conditional distributions

Simulation of complex

geological patterns

• Testing the method

Realiz

atio

n 2

T

rue im

age

Realiz

atio

n 1

Third cum

Third cum

Variograms NSHistograms Training image

Variograms NS

Test - Sparse data and a ‘disoriented’ TI

Sample Data location(85 data)

Training image 2Training image 1True image

Sample Data location

(25 data)

Test - Very sparse data and a ‘disoriented’ TI

39

Simulation of a gold deposit

40

Geological Domains Drillhole data

cdf in each domain Proportion of samples per domain

A gold deposit

3rd-order and 4th- order cumulant maps (data)

A gold deposit

42

Realization Cdf reproduction

55m

75m

Template dimensions:

Order of approximation:

7

20 Realizations

cdfs

Declustered

Data cdf

A gold deposit

Cumulants of Realizations

Declustered

3rd-order

4th-order

A gold deposit

E-Type

P[Au>0.25

ppm]

Au ppm

5

2.5

0

Prob.1

0.5

0

A gold deposit

Some implications for

mine production forecasts

• Open pit mine schedule

Comments:

• Both schedules above are

stochastic

• Schedules are physically different

(and pit limits)

• High-order simulations lead to 40%

higher NPV

• More ore for less waste

Do the methods of modelling uncertainty matter

to mine production scheduling ?

Read is based on high-order simulations (HOSIM)

Blue is based on second-order simulations (SGSIM)

Metal Production

Cumulative NPV

47

Current Research:

High-order stochastic simulation

via statistical learning

in

reproducing kernel Hilbert space

Statistical learning paradigm

• Learning functional dependency from data

• No reliance on parametric models

• Model complexity and generalization

• Minimizing training error complexity

• Minimizing test error generalization

• High-order simulation

• Learning from the input (capture the regularity)

• Adapt to the new data (generalize to the unseen)

• A framework to manipulate the model complexity48

High-order simulation in kernel space

49

Training imageSample data

Replicates

RKHS

Match the high-order

spatial statistics in RKHS

Distribution

Regularize to a “nicer” space

Some References

Minniakhmetov I, Dimitrakopoulos R, Godoy M (2018) High-order spatial

simulation using Legendre-like orthogonal splines. Mathematical

Geosciences, 50(7): 753-780 (Includes source code of related program)

Minniakhmetov I, Dimitrakopoulos R (2016) Joint high-order simulation of

spatially correlated variables using high-order spatial statistics.

Mathematical Geosciences,DOI:10.1007/s11004-016-9662-x

de Carvalho J P, Dimitrakopoulos R, Minniakhmetov I (2019) High-order

block support spatial simulation and application at a gold deposit.

Mathematical Geosciences, DOI 10.1007/s11004-019-09784-x

Mustapha H, Dimitrakopoulos R, Chatterjee S (2011) Geologic

heterogeneity representation using high-order spatial cumulants for

subsurface flow and transport simulations. Water Resources Research, 47,

doi:10.1029/2010WR009515