takashi goda - university of new south wales · richardson extrapolation and higher order qmc...

Richardson extrapolation and higher order QMC

Takashi Goda

School of Engineering, University of Tokyo

MCM 2019

Partly joint with Josef Dick and Takehito Yoshiki

Takashi Goda (U. Tokyo) Extrapolation and HOQMC MCM 2019 1 / 61

Outline

QMC and digital nets/sequences

Classical QMC

Higher order QMC

Richardson extrapolation and QMC

Application 1: Truncation of higher order nets and sequences

Application 2: Extrapolated polynomial lattice rules

Outline

Classical QMC

Higher order QMC

Quasi-Monte Carlo

Approximate

I (f ) =

∫[0,1)s

f (x) dx ,

where f : [0, 1)s → R is integrable.

Choose an N-element point set P ⊂ [0, 1)s and approximate I (f ) by

QP(f ) =1

∑x∈P

f (x).

This is called a quasi-Monte Carlo (QMC) integration over P.

Digital nets in base 2

Let m, n ∈ N := {1, 2, . . .}.Let C1, . . . ,Cs ∈ Fn×m

2 (F2 := {0, 1}, the two-element field).

Denote the dyadic expansion of 0 ≤ h < 2m by

h = (ηm−1 . . . η1η0)2.

For 1 ≤ j ≤ s, let

xj = (0.ξ(j)1 . . . ξ

(j)n 00 . . .)2,

where ξ(j)1...

ξ(j)n

η0...

ηm−1

∈ Fn2.

This gives a point x = (x1, . . . , xs) ∈ [0, 1)s .

Running through all 0 ≤ h < 2m, we get a 2m-element point set Pcalled a digital net over F2.

h = (ηm−1 . . . η1η0)2.

xj = (0.ξ(j)1 . . . ξ

(j)n 00 . . .)2,

where ξ(j)1...

ξ(j)n

η0...

ηm−1

∈ Fn2.

h = (ηm−1 . . . η1η0)2.

xj = (0.ξ(j)1 . . . ξ

(j)n 00 . . .)2,

where ξ(j)1...

ξ(j)n

η0...

ηm−1

∈ Fn2.

h = (ηm−1 . . . η1η0)2.

xj = (0.ξ(j)1 . . . ξ

(j)n 00 . . .)2,

where ξ(j)1...

ξ(j)n

η0...

ηm−1

∈ Fn2.

Digital sequences in base 2

Let C1, . . . ,Cs ∈ FN×N2 , where every column is assumed to have only

finite non-zero elements.

Denote the dyadic expansion of h ∈ N0 := N ∪ {0} by

h = (. . . η1η0)2.

where all but finite number of ηi are 0. For 1 ≤ j ≤ s, let

xj = (0.ξ(j)1 ξ

(j)2 . . .)2,

where ξ(j)1

ξ(j)2...

η0η1...

∈ FN2 .

This way we get an infinite sequence of points S called a digitalsequence over F2.

Some remarks

“Goodness” (undefined) of digital nets/sequences is determined bygenerating matrices C1, . . . ,Cs .

For nets, m determines total number of points, while n determinesprecision of each point.

For sequences, the assumption that every column has only finitenon-zero elements is to ensure that each point has finite precision.

Precision?

For classical QMC where “goodness” is measured by discrepancy, it isusual to set n = m (square matrices) for nets, or considerupper-triangular matrices for sequences.

For higher order QMC where “goodness” is measured by theworst-case error for a class of smooth functions, we need higherprecision. Typically, n = αm for nets (α: dominating mixedsmoothness of functions).

Aim of this talk

is to show that Richardson extrapolation can be used as a technique toreduce necessary precision of higher order QMC from αm to m.

Implication

Possibility to try large values of α w/o suffering from round-off error.

Search space for C1, . . . ,Cs is significantly reduced.

Precision?

Aim of this talk

Implication

Precision?

Aim of this talk

Implication

Outline

Classical QMC

Higher order QMC

Local discrepancy function

For an N-element point set P and y ∈ [0, 1)s , let

∆P(y) :=1

∑x∈P

1x∈[0,y) − λ([0, y)),

where λ denotes the Lebesgue measure.

Discrepancy and Koksma-Hlawka inequality

For 1 ≤ p ≤ ∞, the Lp-discrepancy of P is defined by

Lp(P) :=

(∫[0,1)s

|∆P(y)|p dy

When p = ∞, the obvious modification is needed.

The integration error is bounded by

|QP(f )− I (f )| ≤ VHK(f )L∞(P)

where VHK(f ) is the Hardy-Krause variation of f .

Main focus on classical QMC

How to construct digital nets/sequences with small L∞(P).

Lp(P) :=

(∫[0,1)s

|∆P(y)|p dy

|QP(f )− I (f )| ≤ VHK(f )L∞(P)

Lp(P) :=

(∫[0,1)s

|∆P(y)|p dy

|QP(f )− I (f )| ≤ VHK(f )L∞(P)

Quality measure: t-value

Let t be an integer such that, for any choice d1, . . . , ds ≥ 0 withd1 + · · ·+ ds = m − t,the first d1 rows of C1...the first ds rows of Cs

are linearly independent over F2. P is called a digital (t,m, s)-net.

...C1 C2 Cs

d1d2 ds

An equi-distribution property of digital (t,m, s)-nets: every dyadicelementary box of the form

s∏j=1

[aj2cj

,aj + 1

c1, . . . , cs ≥ 0,

c1 + · · ·+ cs = m − t,

0 ≤ aj < 2cj

contains exactly 2t points.

Let t be an integer such that, for any m ≥ t, the first 2m points of Sare a digital (t,m, s)-net. S is called a digital (t, s)-sequence.

An equi-distribution property of digital (t,m, s)-nets: every dyadicelementary box of the form

s∏j=1

[aj2cj

,aj + 1

c1, . . . , cs ≥ 0,

c1 + · · ·+ cs = m − t,

0 ≤ aj < 2cj

contains exactly 2t points.

Let t be an integer such that, for any m ≥ t, the first 2m points of Sare a digital (t,m, s)-net. S is called a digital (t, s)-sequence.

t-value and star-discrepancy

If P is a digital (t,m, s)-net over F2,

L∞(P) ≤ Cs,t(logN)s−1

Nwith N = 2m.

If P is the first N points of a digital (t, s)-sequence over F2,

L∞(P) ≤ Ds,t(logN)s

for all N ≥ 2.

Explicit construction

For s = 1, C1 can be the identity matrix, which generates the famousdigital (0, 1)-sequence called van der Corput sequence.

For s = 2, C1 and C2 can be

1 0 · · · 00 1 · · · 0...

.... . .

...0 0 · · · 1

0 · · · 0 10 · · · 1 0...

. . ....

...1 · · · 0 0

which generates a digital (0,m, 2)-net, known as the Hammersleypoint set. Not extensible in m.

For s = 1, C1 can be the identity matrix, which generates the famousdigital (0, 1)-sequence called van der Corput sequence.

For s = 2, C1 and C2 can be

1 0 · · · 00 1 · · · 0...

.... . .

...0 0 · · · 1

0 · · · 0 10 · · · 1 0...

. . ....

...1 · · · 0 0

which generates a digital (0,m, 2)-net, known as the Hammersleypoint set. Not extensible in m.

Figure: Hammersley point set for m = 6

Explicit constructionLet p1, p2, . . . ∈ F2[x ] be a sequence of distinct primitive/irreduciblepolynomials over F2 with e1 ≤ e2 ≤ · · · where ej = deg(pj).

For each j , Cj = (c(j)k,l ) is given by the coefficients of the following

Laurent series:

xej−1

pj(x)=

c(j)1,1

c(j)1,2

x2+ · · ·

pj(x)=

c(j)ej ,1

c(j)ej ,2

x2+ · · ·

xej−1

(pj(x))2=

c(j)ej+1,1

c(j)ej+1,2

x2+ · · ·

(Sobol’ 1967; Niederreiter 1988; Tezuka 1993)

Explicit constructionLet p1, p2, . . . ∈ F2[x ] be a sequence of distinct primitive/irreduciblepolynomials over F2 with e1 ≤ e2 ≤ · · · where ej = deg(pj).

For each j , Cj = (c(j)k,l ) is given by the coefficients of the following

Laurent series:

xej−1

pj(x)=

c(j)1,1

c(j)1,2

x2+ · · ·

pj(x)=

c(j)ej ,1

c(j)ej ,2

x2+ · · ·

xej−1

(pj(x))2=

c(j)ej+1,1

c(j)ej+1,2

x2+ · · ·

(Sobol’ 1967; Niederreiter 1988; Tezuka 1993)

All matrices Cj ∈ FN×N2 are upper triangular and generate a digital

(t, s)-sequence with t = (e1 − 1) + · · ·+ (es − 1).(I used a C implementation for this sequence in at least more than 58636dimensions due to Tomohiko Hironaka.)

Figure: 2D projections of the first 26 points of the Niederreiter sequence.

Polynomial lattice point sets (Niederreiter, 1992)

Let p ∈ F2[x ] be irreducible with deg(p) = m

Let q = (q1, . . . , qs) ∈ (F2[x ])s with deg(qj) < m.

For each j , Cj isgiven by the square Hankel matrix

a(j)1 a

(j)2 · · · a

a(j)2 a

(j)3 . . . a

(j)m+1

......

. . ....

a(j)m a

(j)m+1 . . . a

(j)2m−1

∈ Fm×m2

where a(j)1 , a

(j)2 , . . . are the coefficients of the Laurent series

x2+ · · · .

The resulting digital net is called a polynomial lattice point setP(p,q). Usually the vector q is constructed by a (fast) computersearch algorithm (Nuyens & Cools, 2006; P. Kritzer, this afternoon!).

Let q = (q1, . . . , qs) ∈ (F2[x ])s with deg(qj) < m. For each j , Cj is

given by the square Hankel matrix

a(j)1 a

(j)2 · · · a

a(j)2 a

(j)3 . . . a

(j)m+1

......

. . ....

a(j)m a

(j)m+1 . . . a

(j)2m−1

∈ Fm×m2

where a(j)1 , a

x2+ · · · .

a(j)1 a

(j)2 · · · a

a(j)2 a

(j)3 . . . a

(j)m+1

......

. . ....

a(j)m a

(j)m+1 . . . a

(j)2m−1

∈ Fm×m2

where a(j)1 , a

x2+ · · · .

The resulting digital net is called a polynomial lattice point setP(p,q). Usually the vector q is constructed by a (fast) computersearch algorithm (Nuyens & Cools, 2006;

P. Kritzer, this afternoon!).

a(j)1 a

(j)2 · · · a

a(j)2 a

(j)3 . . . a

(j)m+1

......

. . ....

a(j)m a

(j)m+1 . . . a

(j)2m−1

∈ Fm×m2

where a(j)1 , a

x2+ · · · .

Outline

Classical QMC

Higher order QMC

What if f is smooth

In some applications, such as PDEs with random coefficients, f canbe smooth. A proper design of QMC point sets enables higher orderconvergence of the integration error than O(1/N) as expected fromthe KH inequality.

So far, QMC point sets achieving higher order convergence fornon-periodic smooth functions are

1 Higher order digital nets/sequences (Dick, 2008; ...), and

2 Tent-transformed lattice point sets (Hickernell, 2002; ...).

In this talk, I will focus on higher order digital nets/sequences.The contents of this section are mostly developed by Dick (2008).Please refer to a recent review by G. & Suzuki (arXiv:1903.12353).

What if f is smooth

In some applications, such as PDEs with random coefficients, f canbe smooth. A proper design of QMC point sets enables higher orderconvergence of the integration error than O(1/N) as expected fromthe KH inequality.

So far, QMC point sets achieving higher order convergence fornon-periodic smooth functions are

1 Higher order digital nets/sequences (Dick, 2008; ...), and

2 Tent-transformed lattice point sets (Hickernell, 2002; ...).

In this talk, I will focus on higher order digital nets/sequences.The contents of this section are mostly developed by Dick (2008).Please refer to a recent review by G. & Suzuki (arXiv:1903.12353).

Quality measure: high order t-value

Let α ∈ N.Let t be an integer such that, for any choice1 ≤ dj ,vj < · · · < dj ,1 ≤ αm, 0 ≤ vj ≤ αm, 1 ≤ j ≤ s, with

s∑j=1

min(vj ,α)∑i=1

dj ,i = αm − t,

the d1,v1 , . . . , d1,1-th rows of C1...the ds,vs , . . . , ds,1-th rows of Cs

are linearly independent over F2. P is called an order α digital(t,m, s)-net.

Order α digital (t,m, s)-nets hold equi-distribution properties: unionof dyadic elementary boxes contains the fair number of points (shownlater visually).

Let t be an integer such that, for any αm ≥ t, the first 2m points ofS are an order α digital (t,m, s)-net. S is called an order α digital(t, s)-sequence.

Order α digital (t,m, s)-nets hold equi-distribution properties: unionof dyadic elementary boxes contains the fair number of points (shownlater visually).

Let t be an integer such that, for any αm ≥ t, the first 2m points ofS are an order α digital (t,m, s)-net. S is called an order α digital(t, s)-sequence.

Define Dα : [0, 1)α → [0, 1) byx1 = (0.ξ

(1)1 ξ

(1)2 ξ

(1)3 . . .)2

x2 = (0.ξ(2)1 ξ

(2)2 ξ

(2)3 . . .)2

xα = (0.ξ(α)1 ξ

(α)2 ξ

(α)3 . . .)2

7→ (0. ξ(1)1 ξ

(2)1 . . . ξ

(α)1︸︷︷︸

ξ(1)2 ξ

(2)2 . . . ξ

(α)2︸︷︷︸

. . .)2.

For x ∈ [0, 1)αs , let

Dα(x) = (Dα(x1, . . . , xα),Dα(xα+1, . . . , x2α),

. . . ,Dα(xα(s−1)+1, . . . , xαs)) ∈ [0, 1)s .

For a digital (t,m, αs)-net P, we write

Dα(P) = {Dα(x) | x ∈ P}.

Define Dα : [0, 1)α → [0, 1) byx1 = (0.ξ

(1)1 ξ

(1)2 ξ

(1)3 . . .)2

x2 = (0.ξ(2)1 ξ

(2)2 ξ

(2)3 . . .)2

xα = (0.ξ(α)1 ξ

(α)2 ξ

(α)3 . . .)2

7→ (0. ξ(1)1 ξ

(2)1 . . . ξ

(α)1︸︷︷︸

ξ(1)2 ξ

(2)2 . . . ξ

(α)2︸︷︷︸

. . .)2.

Dα(x) = (Dα(x1, . . . , xα),Dα(xα+1, . . . , x2α),

. . . ,Dα(xα(s−1)+1, . . . , xαs)) ∈ [0, 1)s .

Dα(P) = {Dα(x) | x ∈ P}.

Define Dα : [0, 1)α → [0, 1) byx1 = (0.ξ

(1)1 ξ

(1)2 ξ

(1)3 . . .)2

x2 = (0.ξ(2)1 ξ

(2)2 ξ

(2)3 . . .)2

xα = (0.ξ(α)1 ξ

(α)2 ξ

(α)3 . . .)2

7→ (0. ξ(1)1 ξ

(2)1 . . . ξ

(α)1︸︷︷︸

ξ(1)2 ξ

(2)2 . . . ξ

(α)2︸︷︷︸

. . .)2.

Dα(x) = (Dα(x1, . . . , xα),Dα(xα+1, . . . , x2α),

. . . ,Dα(xα(s−1)+1, . . . , xαs)) ∈ [0, 1)s .

Dα(P) = {Dα(x) | x ∈ P}.

Another look at constructionLet P be a digital (t,m, αs)-net with C1, . . . ,Cαs ∈ Fm×m

2 . We write

c(1)1...

c(2)1...

, . . . ,Cα =

c(α)1...

c(α)m

, . . . .

Dα(P) is a digital net with D1, . . . ,Ds ∈ Fαm×m2 where

c(2)1...

c(α)1...

c(α)m

c(α+1)1

c(α+2)1...

c(2α)1...

c(α+1)m

c(α+2)m

c(2α)m

, . . . .

Another look at constructionLet P be a digital (t,m, αs)-net with C1, . . . ,Cαs ∈ Fm×m

2 . We write

c(1)1...

c(2)1...

, . . . ,Cα =

c(α)1...

c(α)m

, . . . .

Dα(P) is a digital net with D1, . . . ,Ds ∈ Fαm×m2 where

c(2)1...

c(α)1...

c(α)m

c(α+1)1

c(α+2)1...

c(2α)1...

c(α+1)m

c(α+2)m

c(2α)m

, . . . .

High order t-value

For a digital (t,m, αs)-net P, Dα(P) is an order α digital(t ′,m, s)-net with

t ′ ≤ αmin

{m, t +

⌊s(α− 1)

For a digital (t, αs)-sequences S, Dα(S) is an order α digital(t ′, s)-sequences with

t ′ ≤ αt +sα(α− 1)

Interlaced Sobol’ point sets with α = 2

Interlaced polynomial lattice point sets

Let p ∈ F2[x ] be irreducible with deg(p) = m, and letq = (q1, . . . , qαs) ∈ (F2[x ])

αs with deg(qj) < m.

An interlaced polynomial lattice point set is just Dα(P(p,q)).

The vector q can be constructed component by component. In thesimplest case, the necessary construction cost is of O(αsN logN) withO(N) memory (G. & Dick, 2015; G., 2015). In applications to PDEswith random coefficients, the criterion sometimes becomes a bitcomplicated, requiring O(αsN logN + α2s2N) construction cost withO(αsN) memory (Dick, Kuo, Le Gia, Nuyens & Schwab, 2014).

Interlaced polynomial lattice point sets

Let p ∈ F2[x ] be irreducible with deg(p) = m, and letq = (q1, . . . , qαs) ∈ (F2[x ])

αs with deg(qj) < m.

An interlaced polynomial lattice point set is just Dα(P(p,q)).

The vector q can be constructed component by component. In thesimplest case, the necessary construction cost is of O(αsN logN) withO(N) memory (G. & Dick, 2015; G., 2015). In applications to PDEswith random coefficients, the criterion sometimes becomes a bitcomplicated, requiring O(αsN logN + α2s2N) construction cost withO(αsN) memory (Dick, Kuo, Le Gia, Nuyens & Schwab, 2014).

Outline

Classical QMC

Higher order QMC

Walsh functions

For k = (. . . κ1κ0)2 ∈ N0, the k-th Walsh function is defined by

walk(x) = (−1)κ0ξ1+κ1ξ2+···,

where x = (0.ξ1ξ2 . . .)2 ∈ [0, 1), unique in the sense that infinitelymany ξi are equal to 0.

For s ≥ 1 and k = (k1, . . . , ks) ∈ Ns0, the k-th Walsh function is

defined by

walk(x) =s∏

walkj (xj).

Walsh functions

For k = (. . . κ1κ0)2 ∈ N0, the k-th Walsh function is defined by

walk(x) = (−1)κ0ξ1+κ1ξ2+···,

where x = (0.ξ1ξ2 . . .)2 ∈ [0, 1), unique in the sense that infinitelymany ξi are equal to 0.

For s ≥ 1 and k = (k1, . . . , ks) ∈ Ns0, the k-th Walsh function is

defined by

walk(x) =s∏

walkj (xj).

Walsh functions

Figure: The k-th Walsh functions for k = 0, 1, 2, 3

Walsh functions

Figure: The k-th Walsh functions for k = 4, 5, 6, 7

Walsh functions

Every Walsh function is a piecewise constant function.

Importantly the system of Walsh functions {walk | k ∈ Ns0} is a

complete orthonormal basis in L2([0, 1)s). Thus we have a Walsh

series of f : ∑k∈Ns

f̂ (k)walk(x)

f̂ (k) =∫[0,1)s

f (x)walk(x) dx .

Walsh functions

Every Walsh function is a piecewise constant function.

Importantly the system of Walsh functions {walk | k ∈ Ns0} is a

complete orthonormal basis in L2([0, 1)s). Thus we have a Walsh

series of f : ∑k∈Ns

f̂ (k)walk(x)

f̂ (k) =∫[0,1)s

f (x)walk(x) dx .

Dual nets

For a digital net P with C1, . . . ,Cs ∈ Fn×m2 , the dual net is defined by

P⊥ =

k ∈ Ns0

∣∣∣C⊤1

κ1,0...

κ1,n−1

⊕ · · · ⊕ C⊤s

κs,0...

κs,n−1

= 0 ∈ Fm2

where we writekj = (. . . κj ,1κj ,0)2.

Trivial fact

k ∈ P⊥ if 2n | k , i.e., 2n | kj for all j = 1, . . . , s.

Dual nets

For a digital net P with C1, . . . ,Cs ∈ Fn×m2 , the dual net is defined by

P⊥ =

k ∈ Ns0

∣∣∣C⊤1

κ1,0...

κ1,n−1

⊕ · · · ⊕ C⊤s

κs,0...

κs,n−1

= 0 ∈ Fm2

where we writekj = (. . . κj ,1κj ,0)2.

Trivial fact

k ∈ P⊥ if 2n | k , i.e., 2n | kj for all j = 1, . . . , s.

Character property

Most of the Walsh functions can be integrated exactly:

∑x∈P

walk(x) =

{1 if k ∈ P⊥,

0 otherwise.

I will also use the following fact:

2n−1∑i=0

∑ξ0,...,ξn−1∈{0,1}

n∏i=1

(−1)κi−1ξi

n∏i=1

∑ξi∈{0,1}

(−1)κi−1ξi

{1 if κ0 = . . . = κn−1 = 0, i.e., if 2n | k,0 otherwise.

Character property

Most of the Walsh functions can be integrated exactly:

∑x∈P

walk(x) =

{1 if k ∈ P⊥,

0 otherwise.

I will also use the following fact:

2n−1∑i=0

∑ξ0,...,ξn−1∈{0,1}

n∏i=1

(−1)κi−1ξi

n∏i=1

∑ξi∈{0,1}

(−1)κi−1ξi

{1 if κ0 = . . . = κn−1 = 0, i.e., if 2n | k,0 otherwise.

Decomposition of integration error

For a digital net P with C1, . . . ,Cs ∈ Fn×m2 , we have

QP(f )− I (f ) =1

∑x∈P

∑k∈Ns

f̂ (k)walk(x)− f̂ (0)

=∑k∈Ns

f̂ (k)1

∑x∈P

walk(x)− f̂ (0)

k∈P⊥\{0}

f̂ (k)

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈P⊥\{0}2n|k

f̂ (k)

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈Ns0\{0}

f̂ (k).

QP(f )− I (f ) =1

∑x∈P

∑k∈Ns

f̂ (k)walk(x)− f̂ (0)

=∑k∈Ns

f̂ (k)1

∑x∈P

walk(x)− f̂ (0)

k∈P⊥\{0}

f̂ (k)

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈P⊥\{0}2n|k

f̂ (k)

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈Ns0\{0}

f̂ (k).

QP(f )− I (f ) =1

∑x∈P

∑k∈Ns

f̂ (k)walk(x)− f̂ (0)

=∑k∈Ns

f̂ (k)1

∑x∈P

walk(x)− f̂ (0)

k∈P⊥\{0}

f̂ (k)

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈P⊥\{0}2n|k

f̂ (k)

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈Ns0\{0}

f̂ (k).

QP(f )− I (f ) =1

∑x∈P

∑k∈Ns

f̂ (k)walk(x)− f̂ (0)

=∑k∈Ns

f̂ (k)1

∑x∈P

walk(x)− f̂ (0)

k∈P⊥\{0}

f̂ (k)

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈P⊥\{0}2n|k

f̂ (k)

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈Ns0\{0}

f̂ (k).

QP(f )− I (f ) =1

∑x∈P

∑k∈Ns

f̂ (k)walk(x)− f̂ (0)

=∑k∈Ns

f̂ (k)1

∑x∈P

walk(x)− f̂ (0)

k∈P⊥\{0}

f̂ (k)

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈P⊥\{0}2n|k

f̂ (k)

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈Ns0\{0}

f̂ (k).

What is the second term?

QP(f )− I (f ) =∑

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈Ns0\{0}

f̂ (k)

The second term is...

∑k∈Ns

0\{0}2n|k

f̂ (k) =∑k∈Ns

f̂ (k)1

2n−1∑i1,...,is=0

, . . . ,is2n

)− f̂ (0)

2n−1∑i1,...,is=0

∑k∈Ns

f̂ (k)walk

, . . . ,is2n

)− I (f )

2n−1∑i1,...,is=0

, . . . ,is2n

)− I (f )

the integration error for a regular grid!

What is the second term?

QP(f )− I (f ) =∑

k∈P⊥\{0}2n∤k

f̂ (k) +∑

k∈Ns0\{0}

f̂ (k)

The second term is...∑k∈Ns

0\{0}2n|k

f̂ (k) =∑k∈Ns

f̂ (k)1

2n−1∑i1,...,is=0

, . . . ,is2n

)− f̂ (0)

2n−1∑i1,...,is=0

∑k∈Ns

f̂ (k)walk

, . . . ,is2n

)− I (f )

2n−1∑i1,...,is=0

, . . . ,is2n

)− I (f )

the integration error for a regular grid!

Digital net as a subset of regular grid

QP(f )− I (f ) =∑

k∈P⊥\{0}2n∤k

f̂ (k) +

2n−1∑i1,...,is=0

, . . . ,is2n

)− I (f )

Figure: Hammersley point set for m = n = 6 with regular grid

Euler-Maclaurin

If f has partial mixed derivatives up to order α in each variable,

2n−1∑i1,...,is=0

, . . . ,is2n

)− I (f )

=c1(f )

2n+ · · ·+ cα−1(f )

2(α−1)n+ O(2−αn),

cτ (f ) =∑

τ1,...,τs≥0τ1+···+τs=τ

I (f (τ1,...,τs))s∏

j=1τj>0

with bi denoting the i-th Bernoulli number divided by i !.

For a detailed proof of this claim, see Dick, G. & Yoshiki (2019).

Euler-Maclaurin

Using this result, we have

QP(f )− I (f ) =∑

k∈P⊥\{0}2n∤k

f̂ (k) +c1(f )

2n+ · · ·+ cα−1(f )

2(α−1)n+ O(2−αn).

Consider high precision, say n = αm. Noting that N = 2m, the firstterm becomes dominant since

QP(f )− I (f ) =∑

k∈P⊥\{0}2n∤k

f̂ (k) +c1(f )

Nα+ · · ·+ cα−1(f )

Nα(α−1)+ O(N−α2

Remark: For functions with dominating mixed smoothness α, theintegration error cannot be better than O(N−α) in general.

Euler-Maclaurin

Using this result, we have

QP(f )− I (f ) =∑

k∈P⊥\{0}2n∤k

f̂ (k) +c1(f )

2n+ · · ·+ cα−1(f )

2(α−1)n+ O(2−αn).

Consider high precision, say n = αm. Noting that N = 2m, the firstterm becomes dominant since

QP(f )− I (f ) =∑

k∈P⊥\{0}2n∤k

f̂ (k) +c1(f )

Nα+ · · ·+ cα−1(f )

Nα(α−1)+ O(N−α2

Remark: For functions with dominating mixed smoothness α, theintegration error cannot be better than O(N−α) in general.

Richardson extrapolation

Consider the case n = m.

Let us simplify the situation for a moment and consider the followingexpansions for successive values of m:

I(1)m−α+1 := c0 +

c12m−α+1

22(m−α+1)+ · · ·+ cα−1

2(α−1)(m−α+1)

I(1)m−1 := c0 +

c12m−1

22(m−1)+ · · ·+ cα−1

2(α−1)(m−1)

I(1)m := c0 +

+c222m

+ · · ·+ cα−1

2(α−1)m

Problem: Estimate c0 from values of I(0)m−α+1, . . . , I

(0)m as precisely as

possible (without knowing c1, . . . , cα−1).

Consider the case n = m.

Let us simplify the situation for a moment and consider the followingexpansions for successive values of m:

I(1)m−α+1 := c0 +

c12m−α+1

22(m−α+1)+ · · ·+ cα−1

2(α−1)(m−α+1)

I(1)m−1 := c0 +

c12m−1

22(m−1)+ · · ·+ cα−1

2(α−1)(m−1)

I(1)m := c0 +

+c222m

+ · · ·+ cα−1

2(α−1)m

Problem: Estimate c0 from values of I(0)m−α+1, . . . , I

(0)m as precisely as

possible (without knowing c1, . . . , cα−1).

Do the following recursion: For τ = 1, . . . , α− 1, compute

I(τ+1)n :=

2τ I(τ)n − I

(τ)n−1

2τ − 1for n = m − α+ τ + 1, . . . ,m

Schematically this recursion goes like

I(1)m−α+1 I

(1)m−α+2 · · · I

(1)m−1 I

↘ ↓ ↘ · · · ↓ ↘ ↓I(2)m−α+2 · · · I

(2)m−1 I

↘ · · · ↓ ↘ ↓. . .

I(α−1)m−1 I

(α−1)m

↘ ↓I(α)m

We will end up with I(α)m = c0. In reality, we have other terms...

Use a set of digital nets

For m − α < n ≤ m, let Pn be a digital net with C1, . . . ,Cs ∈ Fn×n2 .

Here we do NOT require Pm−α+1 ⊂ Pm−α+2 ⊂ · · · ⊂ Pm.

Richardson extrapolation can push out the intermediate terms of theexpansion

QPn(f )− I (f ) =∑

k∈P⊥n \{0}2n∤k

f̂ (k) +c1(f )

2n+ · · ·+ cα−1(f )

2(α−1)n+ O(2−αn)

without knowing c1(f ), . . . , cα−1(f ).

Use a set of digital nets

This way we arrive at an extrapolated QMC rule which satisfies

m∑n=m−α+1

wnQPn(f )− I (f ) =m∑

n=m−α+1

∑k∈P⊥

n \{0}2n∤k

f̂ (k) + O(2−αm),

where wn’s depend only on α,

m∑n=m−α+1

wn = 1 andm∑

n=m−α+1

|wn| ≤α−1∏i=1

2i + 1

2i − 1.

NOTE: The total number of points is 2m−α+1 + · · ·+ 2m < 2m+1.

What we want

m∑n=m−α+1

wnQPn(f )− I (f ) =m∑

n=m−α+1

∑k∈P⊥

n \{0}2n∤k

f̂ (k) + O(2−αm),

If we have digital nets with C1, . . . ,Cs ∈ Fn×n2 which satisfy∑

k∈P⊥n \{0}2n∤k

f̂ (k) = O(2−(α−ε)n) with arbitrarily small ε > 0,

extrapolated QMC rule achieves the almost optimal rate ofconvergence for smooth functions.

Outline

Classical QMC

Higher order QMC

Truncation operator

For x = (0.ξ1ξ2 . . .)2 ∈ [0, 1), define the map

trm(x) := (0.ξ1ξ2 . . . ξm00 . . .)2,

For a point set P ⊂ [0, 1)s , we write

trm(P) = {trm(x) | x ∈ P},

where trm is applied componentwise.

If P is a digital net with C1, . . . ,Cs ∈ Fn×m2 for n ≥ m, trm(P) is a

digital net with upper m ×m submatrices of C1, . . . ,Cs .

What G. (2019+) proves

Theorem

Let P be an order α digital (t,m, s)-net. For functions with dominatingmixed smoothness α ≥ 2, we have∑

k∈(trm(P))⊥\{0}2m∤k

f̂ (k) = O

((logN)αs

with N = 2m.

Compare:∑k∈P⊥\{0}

f̂ (k) = O

((logN)αs

∑k∈(trm(P))⊥\{0}

f̂ (k) = O

Truncation of higher order digital nets

Figure: Interlaced Sobol’ point sets: original (left) and after truncation (right)

Numerical results: original higher order QMCUnivariate test function:

f (x) = x3 (log x + 0.25) .

Interlaced Sobol’ sequences: If αm > 52, tr52 is applied.

0 5 10 15 20 25log

g 2(abs

Order 2 Sobol'Order 3 Sobol'N-2

Numerical results: extrapolation

QP(f )− I (f ) =∑

k∈P⊥\{0}2n∤k

f̂ (k) +c1(f )

2n+ · · ·+ cα−1(f )

2(α−1)n+ O(2−αn).

Extrapolation applied to truncated interlaced Sobol’ sequences: order2 (left) and order 3 (right)

0 5 10 15 20 25log

log 2(a

0 5 10 15 20 25log

log 2(a

Numerical results: comparison for large sTest function:

f (x) =s∏

[1 + γj

(xcj − 1

with s = 100, c = 1.3 and γj = j−2.

0 5 10 15 20log

log 2(a

Order 2 Sobol'N-2

Outline

Classical QMC

Higher order QMC

What Dick, G. & Yoshiki (2019) prove

Theorem

Let p ∈ F2[x ] be irreducible with deg(p) = m. Component-by-componentalgorithm based on a computable criterion can find good q ∈ (F2[x ])

which satisfies∑k∈P(p,q)⊥\{0}

2m∤k

f̂ (k) = O(2−(α−ε)m) with arbitrarily small ε > 0,

for functions with dominating mixed smoothness α.

Here “a computable criterion” has been provided by Baldeaux, Dick,Leobacher, Nuyens & Pillichshammer (2012), which stems from thedecay of Walsh coefficients for smooth functions.

Construction cost

Since we do NOT require Pm−α+1 ⊂ Pm−α+2 ⊂ · · · ⊂ Pm, polynomiallattice point set of each size can be constructed independently.

Construction cost of each is of O((s + α)n2n). In total, the cost willbe of order

m∑n=m−α+1

(s + α)n2n ≤ (s + α)N log2N,

where N = 2m−α+1 + · · ·+ 2m. This is better than interlacedpolynomial lattice point sets for which we need O(αsN logN).

Fast QMC matrix-vector product

In some applications, we want to estimate the integral of the form∫[0,1)s

f (Ax) dx .

In terms of computational cost, computing Ax can be dominant.

Even if A does not have any special structure, this computation canbe made fast by using (polynomial) lattice point sets (Dick, Kuo, LeGia, Schwab, 2015).

This strategy does not work for interlaced polynomial lattice rules,whereas it does work for extrapolated polynomial lattice rules.

Numerical results (α = 2)Test function:

f (x) =s∏

γj1 + γjx2j

with s = 100 and γj = j−2.Extrapolated rule with α = 2 (red), interlaced rule with α = 2 (green)

4 6 8 10 12 14 16

Numerical results (α = 3)Test function:

f (x) =s∏

γj1 + γjx2j

with s = 100 and γj = j−2.Extrapolated rule with α = 3 (red), interlaced rule with α = 3 (green)

4 6 8 10 12 14 16

Conclusions

As far as I know, this is the first attempt to apply an extrapolationtechnique to QMC. Roughly speaking, Richardson extrapolation isshown useful for reducing necessary precision significantly.

Two results are obtained so far1 truncation of higher order digital nets/sequences: high-order

convergence can be recovered by applying Richardson extrapolation.2 introduction of extrapolated polynomial lattice rules: without relying on

interlacing, we can construct high-order convergent QMC-based rules.

Do we have another good application of extrapolation? What aboutlattice rules?

Can we randomize extrapolated QMC rule? Partially yes, if the samerandomly chosen digital shift is applied. What about scrambling?

Conclusions

As far as I know, this is the first attempt to apply an extrapolationtechnique to QMC. Roughly speaking, Richardson extrapolation isshown useful for reducing necessary precision significantly.

Two results are obtained so far1 truncation of higher order digital nets/sequences: high-order

convergence can be recovered by applying Richardson extrapolation.2 introduction of extrapolated polynomial lattice rules: without relying on

interlacing, we can construct high-order convergent QMC-based rules.

Do we have another good application of extrapolation? What aboutlattice rules?

Can we randomize extrapolated QMC rule? Partially yes, if the samerandomly chosen digital shift is applied. What about scrambling?

If you are interested, please refer to

J. Dick, TG, T. Yoshiki: Richardson extrapolation of polynomiallattice rules, SIAM Journal on Numerical Analysis, 57(1):44-69, 2019.

TG: Richardson extrapolation allows truncation of higher order digitalnets and sequences, IMA Journal of Numerical Analysis, online(arXiv:1805.09490).

Thank you for your attention!

takashi goda - university of new south wales · richardson extrapolation and higher order qmc...

Documents