ergodic properties of lévy-driven sdes arising from ... · the hal n-whitt regime," ann....

Ergodic properties of Levy-driven SDEs arising from multiclass

many-server networks, and ergodic control of a class of jump diffusions

Ari Arapostathis

The University of Texas at Austin

Institute of Mathematics and Its ApplicationsQueueing and Networks

May 17, 2018

(Collaborators: Luis Caffarelli, Guodong Pang, Nikola Sandric, and Yi Zheng)

Ari Arapostathis (U.T. Austin) Ergodic properties of Levy-driven SDEs IMA May 17, 2018 1 / 35

Contents

I. Multiclass many-server queueing networks (Halfin–Whitt regime)

Networks under heavy-tailed arrivals and/or service interruptions

Levy driven SDEs

The fractional Laplacian and anisotropic α-stable processes

Subgeometric (subexponential) ergodicity

Lower bounds on rate of convergence

II. Ergodic control of a class of related jump diffusions

Jump diffusions with finite Levy measures and rough kernels

Weak formulation of the ergodic control problem

HJB and verification of optimality via analytical methods

Multiclass queueing networks

A. Arapostathis, A. Biswas, and G. Pang, “Ergodic control of multi-class M/M/N + Mqueues in the Halfin-Whitt regime,” Ann. Appl. Probab. 25 (2015), pp. 3511–3570.

A. Arapostathis, G. Pang, and N. Sandric, “Ergodicity of Levy-driven SDEs arising frommulticlass many-server queues,” arXiv: 1707.09674.

A. Arapostathis, L. Caffarelli, G. Pang, and Y. Zheng “Ergodic control of a class of jumpdiffusions with finite Levy measures and rough kernels,” arXiv: 1801.07669.

Multiclass queueing networks

A. Arapostathis, A. Biswas, and G. Pang, “Ergodic control of multi-class M/M/N + Mqueues in the Halfin-Whitt regime,” Ann. Appl. Probab. 25 (2015), pp. 3511–3570.

A. Arapostathis, G. Pang, and N. Sandric, “Ergodicity of Levy-driven SDEs arising frommulticlass many-server queues,” arXiv: 1707.09674.

A. Arapostathis, L. Caffarelli, G. Pang, and Y. Zheng “Ergodic control of a class of jumpdiffusions with finite Levy measures and rough kernels,” arXiv: 1801.07669.

Ergodic control of multiclass multi-pool (Halfin–Whitt regime)

A. Arapostathis and G. Pang, “Ergodic diffusion control of multiclass multi-pool networks inthe Halfin-Whitt regime,” Ann. Appl. Probab. 26 (2016), pp. 3110–3153.

A. Arapostathis and G. Pang, “Infinite horizon average optimality of the N-network in theHalfin–Whitt regime,” Math. Oper. Res. (2018) (articles in advance).

A. Arapostathis and G. Pang, “Infinite horizon asymptotic average optimality for large-scaleparallel server networks,” Stochastic Process. Appl. (2018) (in press).

Ergodic control of multiclass multi-pool (Halfin–Whitt regime)

A. Arapostathis and G. Pang, “Ergodic diffusion control of multiclass multi-pool networks inthe Halfin-Whitt regime,” Ann. Appl. Probab. 26 (2016), pp. 3110–3153.

A. Arapostathis and G. Pang, “Infinite horizon average optimality of the N-network in theHalfin–Whitt regime,” Math. Oper. Res. (2018) (articles in advance).

A. Arapostathis and G. Pang, “Infinite horizon asymptotic average optimality for large-scaleparallel server networks,” Stochastic Process. Appl. (2018) (in press).

Multiclass many server networks with busty arrivals

G/M/n + M queues with d classes of customers, exponential service andabandonment rates, µi and γi .

Arrival Process: Ani , i = 1, . . . , d , with arrival rate λn

i , and mutually independent.

Define Ani := n−1/α(An

i − λni $), where $(t) ≡ t for each t ≥ 0, and α ∈ (1, 2].

Assume that the arrival processes satisfy an FCLT

An ⇒ A = (A1, . . . ,Ad)′ in (Dd ,M1), as n→∞,

with Ai mutually independent symmetric α-stable processes with Ai (0) ≡ 0,

(Modified) Halfin-Whitt regime:

λni/n → λi > 0, and ˆn

i := n−1/α(λn

i − nλi ) → `i ∈ R ,

With ρn :=∑d

nµithe aggregate traffic intensity, we have

n1−1/α(1− ρn) → ρ = −d∑

as n→∞ ,

where ρn :=∑d

nµiis the aggregate traffic intensity.

Multiclass many server networks with busty arrivals

G/M/n + M queues with d classes of customers, exponential service andabandonment rates, µi and γi .

Arrival Process: Ani , i = 1, . . . , d , with arrival rate λn

i , and mutually independent.

Define Ani := n−1/α(An

i − λni $), where $(t) ≡ t for each t ≥ 0, and α ∈ (1, 2].

Assume that the arrival processes satisfy an FCLT

An ⇒ A = (A1, . . . ,Ad)′ in (Dd ,M1), as n→∞,

with Ai mutually independent symmetric α-stable processes with Ai (0) ≡ 0,

(Modified) Halfin-Whitt regime:

λni/n → λi > 0, and ˆn

i := n−1/α(λn

i − nλi ) → `i ∈ R ,

With ρn :=∑d

nµithe aggregate traffic intensity, we have

n1−1/α(1− ρn) → ρ = −d∑

as n→∞ ,

where ρn :=∑d

nµiis the aggregate traffic intensity.

Multiclass many server networks with busty arrivals (cont.)

X ni (t): the number of class-i customers

Diffusion-scaled process: X ni := n−1/α(X n

i − ρin)

Theorem (SDE limit)

Under a fixed constant scheduling control v ∈ ∆ (= set of probability vectors of

dimension d), provided there exists X (0) such that X n(0)⇒ X (0) as n→∞, then wehave

X n ⇒ X in (Dd ,M1) as n→∞ ,

where the limit process X is a unique strong solution to the SDE

dX (t) = b(X (t), v) dt + dA(t)− σα dW (t) ,

with an initial condition X (0). The drift b(x , v) : Rd ×∆→ Rd takes the form

b(x , v) = `− R(x − 〈e, x〉+v)− 〈e, x〉+Γ v ,

with e = (1, . . . , 1)′ ∈ Rd , R = diag(µ1, . . . , µd), and Γ = diag(γ1, . . . , γd). Also, A isthe limit of the arrival process, W is a standard d-dimensional Brownian motion,independent of A, and σασ

′α = diag(λ1, . . . , λd) if α = 2, while σα = 0 if α ∈ (1, 2).

Multiclass many server networks with service interruptions

Multiclass G/M/n + M queues in the same renewal alternating (up-down, or on-off)random environment, where all the classes of customers are affected simultaneously.

Assume that the system functions normally during up time periods, and all serversstop functioning during down periods, while customers continue entering the systemand may abandon while waiting in queue and those that have started service willwait for the system to resume.

Let {(unk , d

nk ) : k ∈ N} be a sequence of i.i.d. positive random vectors representing the

up-down cycles. Assume that{(un

k , n1/αdn

k ) : k ∈ N}⇒{

(uk , dk) : k ∈ N}

in (R2)∞ as n→∞ ,

where (uk , dk), k ∈ N, are i.i.d. positive random vectors and α ∈ (1, 2].Let N(t) := max{k ≥ 0: Tk ≤ t}, t ≥ 0, with Tk :=

∑ki=1 ui for k ∈ N, and T0 ≡ 0.

Assume that the process {N(t)}t≥0 is Poisson.Similarly, we obtain a limit

dX (t) = b(X (t), v)dt + dA(t)− σα dW (t) + c dJ(t) .

Also, c = (λ1, . . . , λd)′, and the process J is a compound Poisson process, defined by

J(t) :=

N(t)∑k=1

dk , t ≥ 0 .

Let {(unk , d

k , n1/αdn

k ) : k ∈ N}⇒{

(uk , dk) : k ∈ N}

in (R2)∞ as n→∞ ,

Assume that the process {N(t)}t≥0 is Poisson.

Similarly, we obtain a limit

dX (t) = b(X (t), v)dt + dA(t)− σα dW (t) + c dJ(t) .

J(t) :=

N(t)∑k=1

dk , t ≥ 0 .

Let {(unk , d

k , n1/αdn

k ) : k ∈ N}⇒{

(uk , dk) : k ∈ N}

in (R2)∞ as n→∞ ,

Assume that the process {N(t)}t≥0 is Poisson.Similarly, we obtain a limit

dX (t) = b(X (t), v) dt + dA(t)− σα dW (t) + c dJ(t) .

J(t) :=

N(t)∑k=1

dk , t ≥ 0 .

A class of piecewise linear Levy-driven SDEs

Consider a d-dimensional stochastic differential equation (SDE) of the form

dX (t) = b(X (t))dt + σ(X (t)) dW (t) + dL(t), X (0) = x ∈ Rd ,

the function b : Rd → Rd is given by

b(x) = `−M(x − 〈e, x〉+v)− 〈e, x〉+Γ v =

(M + (Γ −M)ve′

)x , e′x > 0 ,

`−Mx , e′x ≤ 0 ,

Here, M ∈ Rd×d is a nonsingular M-matrix1 such that the vector e′M hasnonnegative components.

{W (t)}t≥0 is a standard n-dimensional Brownian motion, and the covariancefunction σ : Rd → Rd×n is locally Lipschitz and satisfies, for some constant κ > 0,

‖σ(x)‖2 ≤ κ(1 + |x |2) , x ∈ Rd ;

{L(t)}t≥0 is a d-dimensional pure-jump Levy process determined by a drift ϑ ∈ Rd

and Levy measure ν(dy).

We also consider Markov controls v : Rd → ∆.

1i.e., M = sI − N, with σ(N) < s, N nonnegative

Two important parameters1. Heaviness of the tail of the Levy measure:

Θc :={θ > 0 :

|y |θν(dy) < ∞}, and θc := sup

{θ ∈ Θc

For α-stable, Θc = (0, α), and θc = α.

2. Effective spare capacity: % := −⟨e,M−1 ˜

⟩, where

`+ ϑ+∫Bc yν(dy) , if

∫Bc |y |ν(dy) <∞

`+ ϑ, otherwise.

Theorem (Necessary and sufficient conditions)

Under no abandonment, or more generally, if the control assigns higher priority to queuesthat do not abandon (Γ v = 0), the conditions % > 0 and 1 ∈ Θc are necessary andsufficient for the process {X (t)}t≥0 to have an invariant probability measure π undersome Markov control. Moreover, if % < 0, then {X (t)}t≥0 is always transient.a Inaddition,

〈e, x〉− π(dx) .

aFor some reason this is hard to prove for α-stable.

Θc :={θ > 0 :

{θ ∈ Θc

⟩, where

∫Bc |y |ν(dy) <∞

`+ ϑ, otherwise.

〈e, x〉− π(dx) .

Θc :={θ > 0 :

{θ ∈ Θc

⟩, where

∫Bc |y |ν(dy) <∞

`+ ϑ, otherwise.

〈e, x〉− π(dx) .

The fractional Laplacian

A naive interpretation: Start with a symmetric density ϕ(y) ∝ |y |−(d+α) on Rd ,α ∈ (0, 2), and consider a ‘long-jump’ random walk on the lattice Zn dictated by thisdensity. Scaling the space as εZn, and the time interval τ = εα, and noting thatϕ(k)τ

= εdϕ(εk), k ∈ Zd , end letting ε↘ 0, we end up with the limit for the transitiondensity

∂tp(t, x) =

p(t, x + y)− p(t, x)

|y |d+αdy .

This transition density behaves as

p(t, x , y) = p(t, y − x) ∼ t

|y − x |d+α∧ t−

d/α .

In this manner we obtain the fractional Laplacian operator

−(−∆)

α/2u(x) := limε↘0

C(d , α)

∫Rd\Bε

u(x + y)− u(x)

|y |d+αdy

= C(d , α)

u(x + y) + u(x − y)− 2u(x)

|y |d+αdy ,

expressed as the principal value of a singular integral, or as an integral of a second orderincremental quotient.

The fractional Laplacian

A naive interpretation: Start with a symmetric density ϕ(y) ∝ |y |−(d+α) on Rd ,α ∈ (0, 2), and consider a ‘long-jump’ random walk on the lattice Zn dictated by thisdensity. Scaling the space as εZn, and the time interval τ = εα, and noting thatϕ(k)τ

= εdϕ(εk), k ∈ Zd , end letting ε↘ 0, we end up with the limit for the transitiondensity

∂tp(t, x) =

p(t, x + y)− p(t, x)

|y |d+αdy .

This transition density behaves as

p(t, x , y) = p(t, y − x) ∼ t

|y − x |d+α∧ t−

d/α .

In this manner we obtain the fractional Laplacian operator

−(−∆)

α/2u(x) := limε↘0

C(d , α)

∫Rd\Bε

u(x + y)− u(x)

|y |d+αdy

= C(d , α)

u(x + y) + u(x − y)− 2u(x)

|y |d+αdy ,

expressed as the principal value of a singular integral, or as an integral of a second orderincremental quotient.

The fractional Laplacian (cont.)

Note that for f ∈ C2(Rd), this can also be computed as

−(−∆)

α/2u(x) = C(d , α)

∫Rd\{0}

(f (x + y)− f (x)− 1B(y)〈y ,∇f (x)〉

The normalization constant C(d , α) ≈ d(2− α) is such that

−(−∆)

α/2u(x) −−−→α↗2

∆u(x) .

Roughly speaking the fractional Laplacian acts like a derivative of order α.

As a result, when α > 1 (subcritical) it is a higher order operator compared to thedrift, while if α < 1 (supercritical) the opposite is the case.

The fractional Laplacian (cont.)

Note that for f ∈ C2(Rd), this can also be computed as

−(−∆)

α/2u(x) = C(d , α)

∫Rd\{0}

(f (x + y)− f (x)− 1B(y)〈y ,∇f (x)〉

The normalization constant C(d , α) ≈ d(2− α) is such that

−(−∆)

α/2u(x) −−−→α↗2

∆u(x) .

Roughly speaking the fractional Laplacian acts like a derivative of order α.

As a result, when α > 1 (subcritical) it is a higher order operator compared to thedrift, while if α < 1 (supercritical) the opposite is the case.

Anisotropic α-stable kernelFor a system under heavy-tailed arrivals, the driving process is an anisotropic Levy processwith independent one-dimensional symmetric α-stable components, i.e., generated by

Lf (x) = (2− α)d∑

∫R∗

d1f (x ; yiei )ηi dyi|yi |1+α

∫Rd∗

d1f (x ; y) ν(dy) ,

where ηi ’s are constants, and

d1f (x ; y) := f (x + y)− f (x)− 1B(y)⟨y ,∇f (x)

⟩, f ∈ C 2(Rd) .

As a result, the Levy measure is highly singular, and lacks the regularity of the isotropicα-stable. In particular, as shown in Bass & Chen (2006, 2010), the Harnack inequalityfails for this operator.

Nevertheless, we have shown the following.

Theorem (irreducibility)

Suppose {L(t)}t≥0 is of the form L(t) = L1(t) + L2(t), t ≥ 0, where {L1(t)}t≥0 and{L2(t)}t≥0 are independent d-dimensional pure-jump Levy processes, such that{L1(t)}t≥0 is an anisotropic Levy process with independent symmetric one-dimensionalα-stable components for α ∈ (0, 2), and {L2(t)}t≥0 is a compound Poisson process.Then the solution {X (t)}t≥0 of the SDE is open-set irreducible and aperiodic.

Anisotropic α-stable kernelFor a system under heavy-tailed arrivals, the driving process is an anisotropic Levy processwith independent one-dimensional symmetric α-stable components, i.e., generated by

Lf (x) = (2− α)d∑

∫R∗

d1f (x ; yiei )ηi dyi|yi |1+α

∫Rd∗

d1f (x ; y) ν(dy) ,

where ηi ’s are constants, and

d1f (x ; y) := f (x + y)− f (x)− 1B(y)⟨y ,∇f (x)

⟩, f ∈ C 2(Rd) .

As a result, the Levy measure is highly singular, and lacks the regularity of the isotropicα-stable. In particular, as shown in Bass & Chen (2006, 2010), the Harnack inequalityfails for this operator.

Nevertheless, we have shown the following.

Theorem (irreducibility)

Suppose {L(t)}t≥0 is of the form L(t) = L1(t) + L2(t), t ≥ 0, where {L1(t)}t≥0 and{L2(t)}t≥0 are independent d-dimensional pure-jump Levy processes, such that{L1(t)}t≥0 is an anisotropic Levy process with independent symmetric one-dimensionalα-stable components for α ∈ (0, 2), and {L2(t)}t≥0 is a compound Poisson process.Then the solution {X (t)}t≥0 of the SDE is open-set irreducible and aperiodic.

Extended generator

In summary, the extended generator of {X (t)}t≥0 takes the form

Af (x) =1

2Tr(a(x)∇2f (x)

)+⟨b(x) + ϑ,∇f (x)

∫Rd∗

d1f (x ; y)ν(dy) ,

with ∇2 denoting the Hessian of f .

Here ν is the Levy measure ν(dy). This is a σ-finite measure on Rd∗ := Rd \ {0}

satisfying∫Rd∗

(1 ∧ |y |2) ν(dy) <∞.

Recall that

d1f (x ; y) := f (x + y)− f (x)− 1B(y)⟨y ,∇f (x)

⟩, f ∈ C 2(Rd) .

and definedf (x ; y) := f (x + y)− f (x)−

⟨y ,∇f (x)

Write ∫Rd∗

d1f (x ; y)ν(dy) =

∫Rd∗

df (x ; y) ν(dy)︸︷︷︸Jν f (x)

⟨ (∫Bc

yν(dy)

)︸︷︷︸

we absorb this in the drift

,∇f (x)

assuming from now on that 1 ∈ Θc .

Extended generator

In summary, the extended generator of {X (t)}t≥0 takes the form

Af (x) =1

2Tr(a(x)∇2f (x)

)+⟨b(x) + ϑ,∇f (x)

∫Rd∗

d1f (x ; y)ν(dy) ,

with ∇2 denoting the Hessian of f .

Here ν is the Levy measure ν(dy). This is a σ-finite measure on Rd∗ := Rd \ {0}

satisfying∫Rd∗

(1 ∧ |y |2) ν(dy) <∞.

Recall that

d1f (x ; y) := f (x + y)− f (x)− 1B(y)⟨y ,∇f (x)

⟩, f ∈ C 2(Rd) .

and definedf (x ; y) := f (x + y)− f (x)−

⟨y ,∇f (x)

Write ∫Rd∗

d1f (x ; y)ν(dy) =

∫Rd∗

df (x ; y) ν(dy)︸︷︷︸Jν f (x)

⟨ (∫Bc

yν(dy)

)︸︷︷︸

we absorb this in the drift

,∇f (x)

assuming from now on that 1 ∈ Θc .

Extended generator (cont.)

After this rearrangement, the constant term in the drift is ˜ (we rename the drift asb). Recall that % = −

⟨e,M−1 ˜

⟩is the (effective) spare capacity.

Some important properties of the operator Jν .Two useful expansions of the singular integral over a domain D ⊂ Rd :∫

D\{0}df (x ; y) ν(dy) =

∫D\{0}

(∫ 1

(1− t)⟨y ,∇2f (x + ty)y

)ν(dy)

∫D\{0}

(∫ 1

⟨y ,∇f (x + ty)−∇f (x)

)ν(dy) .

Recall the definition

Θc :={θ > 0 :

{θ ∈ Θc

So if θc ∈ (0,∞), then Θc = (0, θc), or Θc = (0, θc ].

From the above expansion of the integral we have1 Jν acts like a derivative of order θc (slightly better than that).

2 If f is convex then Jν f ≥ 0.

⟨e,M−1 ˜

D\{0}df (x ; y) ν(dy) =

∫D\{0}

(∫ 1

(1− t)⟨y ,∇2f (x + ty)y

)ν(dy)

∫D\{0}

(∫ 1

⟨y ,∇f (x + ty)−∇f (x)

)ν(dy) .

Θc :={θ > 0 :

{θ ∈ Θc

⟨e,M−1 ˜

D\{0}df (x ; y) ν(dy) =

∫D\{0}

(∫ 1

(1− t)⟨y ,∇2f (x + ty)y

)ν(dy)

∫D\{0}

(∫ 1

⟨y ,∇f (x + ty)−∇f (x)

)ν(dy) .

Θc :={θ > 0 :

{θ ∈ Θc

More precisely:

If θ ∈ Θc , and f ∈ C 2(Rd) satisfies

sup|x|≥1

|x |1−θ max(|∇f (x)|, |x | ‖∇2f (x)‖

)< ∞ ,

then the function Jν f (x) vanishes at infinity when θ ∈ [1, 2), andx 7→ (1 + |x |)2−θ Jν f (x) is bounded when θ ≥ 2.

Common Lyapunov functionsRecall that if Γ v = 0, then

b(x) =

{˜−M(I− ve′)x , e′x > 0 ,

˜−Mx , e′x ≤ 0 ,

Dieker & Gao (2013) show that ∃ a positive definite matrix Q such that

QM + M ′Q � 0 , and QM(I− ve′) + (I− ev ′)M ′Q � 0 .

Using this, they assert positive recurrence in the Brownian case, but do not study therate of convergence (which in the Brownian case it turns out to be exponential).

Employing VQ,θ := 〈x ,Qx〉θ/2, with θ ≥ 1, as Lyapunov function, after some carefulcalculations, we obtain⟨

b(x),∇VQ,θ(x)⟩≤ κ01B(x)− κ1 VQ,θ(x)1Kc

δ(x)− κ1 VQ,θ−1(x)1Kδ (x) .

Here, Kδ is the convex cone Kδ :={x ∈ Rd : 〈e, x〉 > δ|x |

}, for some δ > 0.

We have already seen that JνVQ,θ vanishes at infinity, if θ ∈ Θc .

Assuming that a(x) has sublinear growth, then Tr(a(x)∇2f (x)

)grows slower than

|x |θ−1, so it is small compared to⟨b(x),∇VQ,θ(x)

Combining these we have

AVQ,θ(x) ≤ c0(θ)− c1VQ,θ(x)1Kcδ

(x)− c1VQ,θ−1(x)1Kδ (x) .

Here, c0(θ), c1, and δ are constants.

b(x) =

{˜−M(I− ve′)x , e′x > 0 ,

˜−Mx , e′x ≤ 0 ,

δ(x)− κ1 VQ,θ−1(x)1Kδ (x) .

}, for some δ > 0.

)grows slower than

(x)− c1VQ,θ−1(x)1Kδ (x) .

b(x) =

{˜−M(I− ve′)x , e′x > 0 ,

˜−Mx , e′x ≤ 0 ,

δ(x)− κ1 VQ,θ−1(x)1Kδ (x) .

}, for some δ > 0.

)grows slower than

(x)− c1VQ,θ−1(x)1Kδ (x) .

Brief review of subgeometric ergodicity

Early work: Stone & Waigner (1967), Orey (1971), Lindvall (1979), Ney (1981), Nummelin& Tuominen (1983), Tuominen & Tweedie (1994)

Foster–Lyapunov criteria: Connor & Fort (2009), Douc, Fort, and Moulines (2004, 2009),Fort & Roberts (2005), Locherbach (2016)

Lyapunov–Poincare inequalities: Bakry, Cattiaux, and Guillin (2008), Cattiaux, Guillin, F.Y.Wang, and L. Wu (2009)

The following result is obtained via Foster–Lyapunov criteria.

A simple upper bound

Assuming irreducibility for some skeleton chain, then the Foster–Lyapunov equation

AV (x) ≤ κ1K (x)− V ρ(x) ,

where ρ ∈ (0, 1), κ is a constant, and K is a closed petite set, implies that∥∥Pt(x , · )− π( · )∥∥

TV≤ C t

−ρ1−ρV (x) .

Brief review of subgeometric ergodicity

Early work: Stone & Waigner (1967), Orey (1971), Lindvall (1979), Ney (1981), Nummelin& Tuominen (1983), Tuominen & Tweedie (1994)

Foster–Lyapunov criteria: Connor & Fort (2009), Douc, Fort, and Moulines (2004, 2009),Fort & Roberts (2005), Locherbach (2016)

Lyapunov–Poincare inequalities: Bakry, Cattiaux, and Guillin (2008), Cattiaux, Guillin, F.Y.Wang, and L. Wu (2009)

The following result is obtained via Foster–Lyapunov criteria.

A simple upper bound

Assuming irreducibility for some skeleton chain, then the Foster–Lyapunov equation

AV (x) ≤ κ1K (x)− V ρ(x) ,

where ρ ∈ (0, 1), κ is a constant, and K is a closed petite set, implies that∥∥Pt(x , · )− π( · )∥∥

TV≤ C t

−ρ1−ρV (x) .

Upper bounds on the rate of convergence

Theorem (upper bound)

For any v ∈ ∆ (with Γ v = 0), the process {X (t)}t≥0 admits a unique invariantprobability measure π ∈ P(Rd), and∥∥Pt(x , · )− π( · )

∥∥TV≤ C2(ε) (t ∨ 1)1+ε−θc |x |θc−ε , ε ∈ (0, θc − 1) .

If Γ v 6= 0, and M is diagonal, then we obtain the Foster–Lyapunov equation

AVQ,θ(x) ≤ c0(θ)− c1VQ,θ(x) ,

which implies exponential ergodicity.

Upper bounds on the rate of convergence

Theorem (upper bound)

For any v ∈ ∆ (with Γ v = 0), the process {X (t)}t≥0 admits a unique invariantprobability measure π ∈ P(Rd), and∥∥Pt(x , · )− π( · )

∥∥TV≤ C2(ε) (t ∨ 1)1+ε−θc |x |θc−ε , ε ∈ (0, θc − 1) .

If Γ v 6= 0, and M is diagonal, then we obtain the Foster–Lyapunov equation

AVQ,θ(x) ≤ c0(θ)− c1VQ,θ(x) ,

which implies exponential ergodicity.

Turning the page

Up to now, all this involves known techniques, intertwined with some elaboratecalculations. Augmented by some other results, including convergence inWasserstein distance, this work was submitted to a journal (minus some necessaryand sufficient conditions already discussed).

Then a reviewer asked: This is only an upper bound. Can you show that the processcan’t be exponentially ergodic?

The initial reaction was: This is easy! All that one needs is to show that therecurrence times to an open ball do not have exponential moments, i.e. E[eετ] =∞,for all ε > 0. And indeed, this can be shown.

But then we changed the question to: What is the tightest lower bound one canget?

Turning the page

Lower bounds on the rate of convergence

Hardly anything available in the literature,

except for a nice ‘recipe’ from Martin Hairer(http://www.hairer.org/notes/Convergence.pdf).

Theorem (Hairer)

Let Xt be a Markov process on a Polish space X with invariant measure π, and letG : X → [1,∞) be such that

∃ f : [1,∞)→ [0, 1] such that s f (s)↗∞ as s →∞, andπ({x : G(x) ≥ s}) ≥ f (s) for all s.

∃ g(x , t) : X × R+ → [1,∞), increasing in its second argument, such thatEx

[G(Xt) | X0 = x0

]≤ g(x0, t).

Then, with y(t) a solution of y(t)f(y(t)

)= 2g(x0, t), we have∥∥Pt(x0, · )− π∥∥

TV≥ 1

2f(y(t)

The proof is only one line long, using the definition of the total variation distanceand Markov’s inequality. So one might be inclined to discount it as giving veryconservative estimates.

Theorem (Hairer)

[G(Xt) | X0 = x0

]≤ g(x0, t).

)= 2g(x0, t), we have∥∥Pt(x0, · )− π∥∥

TV≥ 1

2f(y(t)

Theorem (Hairer)

[G(Xt) | X0 = x0

]≤ g(x0, t).

)= 2g(x0, t), we have∥∥Pt(x0, · )− π∥∥

TV≥ 1

2f(y(t)

Theorem (Hairer)

[G(Xt) | X0 = x0

]≤ g(x0, t).

)= 2g(x0, t), we have∥∥Pt(x0, · )− π∥∥

TV≥ 1

2f(y(t)

Another look at the Foster–Lyapunov equation

A|x |θ ≤ c0(θ)− c1|x |θ1Kcδ

(x)− c1|x |θ−11Kδ (x) .

1 With G(x) ∼ |x |θ for θ < α, we can estimate Ex [G(X (t)].

2 The difficult part is to obtain an estimate for the tail of π.

3 The answer to this: Move in the direction of the spare capacity: i.e., choose insteadG(x) = 〈e,M−1x〉α−ε.

Lower bounds for the Levy driven SDE

Theorem (lower bounds)

The diffusion limit process {X (t)}t≥0 is polynomially ergodic, and its rate of convergenceis r(t) ≈ tθc−1. In particular, in the case of an α-stable process (isotropic or not), weobtain the following quantitative bounds. There exist positive constants C1, and C2(ε)such that for all ε ∈ (0, α− 1), we have

+ |x |α−ε) 1−α

1−ε ≤∥∥Pt(x , · )− π( · )

∥∥TV≤ C2(ε)(t ∨ 1)1+ε−α|x |α−ε

for all t > 0, and all x ∈ Rd .

In the case of the Levy process, there exists a positive constant C3(ε) such that for allε ∈ (0, 1

3), and all x ∈ Rd , we have

C3(ε)(tn + |x |θc−ε

)− θc−1+2ε1−3ε ≤

∥∥Pt(x , · )− π( · )∥∥

TV≤ C2(ε)(t ∨ 1)1+ε−θc |x |θc−ε .

for some sequence {tn}n∈N ⊂ [0,∞), tn →∞, depending on x .

Lower bounds for the Levy driven SDE

Theorem (lower bounds)

The diffusion limit process {X (t)}t≥0 is polynomially ergodic, and its rate of convergenceis r(t) ≈ tθc−1. In particular, in the case of an α-stable process (isotropic or not), weobtain the following quantitative bounds. There exist positive constants C1, and C2(ε)such that for all ε ∈ (0, α− 1), we have

+ |x |α−ε) 1−α

1−ε ≤∥∥Pt(x , · )− π( · )

∥∥TV≤ C2(ε)(t ∨ 1)1+ε−α|x |α−ε

for all t > 0, and all x ∈ Rd .

In the case of the Levy process, there exists a positive constant C3(ε) such that for allε ∈ (0, 1

3), and all x ∈ Rd , we have

C3(ε)(tn + |x |θc−ε

)− θc−1+2ε1−3ε ≤

∥∥Pt(x , · )− π( · )∥∥

TV≤ C2(ε)(t ∨ 1)1+ε−θc |x |θc−ε .

for some sequence {tn}n∈N ⊂ [0,∞), tn →∞, depending on x .

Polynomial ergodicity is rather generic

Abstracting the structural property of the drift under no abandonment, we obtainthe same results for general drifts.

Corollary (general drifts)

Consider the following structural hypotheses (for a positive symmetric matrix Q).

(H1) lim sup|x|→∞〈b(x),Qx〉|x|1+θ < 0, for some θ ∈ [0, 1), and a(x) has slower than polynomial

growth of order 1 + θ.

(H2) For some constant γ ∈ [0, 1), one of the following hold.

(i) There exists some x0 ∈ Rd and a positive constant C , such that

〈z0, b(x)〉 ≥ −C(1 + 〈z0, x〉γ

), 〈z0, x〉 ≥ 0 .

(ii) There exists a positive constant C , such that

〈Qx , b(x)〉 ≥ −C(1 + |x |1+γ

), x ∈ Rd .

+ |x |α−ε) 1−γ−α

1−γ−ε ≤∥∥δxPX

t (dy)− π(dy)∥∥

TV≤ C2(ε) (t ∨ 1)

1+ε−α−θ1−θ |x |α−ε .

Note that necessarily γ ≥ θ by hypothesis.

Polynomial ergodicity is rather generic

Abstracting the structural property of the drift under no abandonment, we obtainthe same results for general drifts.

Corollary (general drifts)

Consider the following structural hypotheses (for a positive symmetric matrix Q).

(H1) lim sup|x|→∞〈b(x),Qx〉|x|1+θ < 0, for some θ ∈ [0, 1), and a(x) has slower than polynomial

growth of order 1 + θ.

(H2) For some constant γ ∈ [0, 1), one of the following hold.

(i) There exists some x0 ∈ Rd and a positive constant C , such that

〈z0, b(x)〉 ≥ −C(1 + 〈z0, x〉γ

), 〈z0, x〉 ≥ 0 .

(ii) There exists a positive constant C , such that

〈Qx , b(x)〉 ≥ −C(1 + |x |1+γ

), x ∈ Rd .

+ |x |α−ε) 1−γ−α

1−γ−ε ≤∥∥δxPX

t (dy)− π(dy)∥∥

TV≤ C2(ε) (t ∨ 1)

1+ε−α−θ1−θ |x |α−ε .

Note that necessarily γ ≥ θ by hypothesis.

Existence of moments

I. Systems with no abandonment

Under any Markov control,∫Rd |x |θ−1 π(dx) =∞ for θ /∈ Θc .

This implies that under heavy-tailed arrivals there exists no stabilizing control.

II. Systems with abandonment

Under any Markov control,∫Rd |x |θ π(dx) =∞ for θ /∈ Θc .

Also under any constant Markov control,∫Rd |x |θ π(dx) <∞ for all θ ∈ Θc .

This implies that under heavy-tailed arrivals the invariant measure has no secondmoments.

A conjecture

Provided % > 0 and 1 ∈ Θc , the process {X (t)}t≥0 is polynomially ergodic under anyMarkov control.

For the Markovian “V” model, Gamarnik & Stolyar (2012) have establishedgeometric ergodicity for the prelimit, under any work conserving scheduling policy.

A conjecture

Ergodic control of a related class of jump diffusions

In the study of ergodic control problems there is a certain gap between thestochastic control and PDE communities:

The PDE community is investigating equations of the ‘ergodic type’, sometimeswithout connecting them to a stochastic interpretation, while the stochastic controlcommunity is often restricting the class of admissible controls to guarantee that themartingale problem is well posed, thus restricting the breadth of optimality.

A way to bridge this gap is to address the primal optimization problem, overinfinitesimal ergodic measures.

This has been considered in Fleming & Vermes (1989) for the discounted cost,Stockbridge (1990), Bhatt & Borkar (1996).

Given the recent advances in PDE theory and techniques, we revisit the linearprogramming formulation.

The goal is to state the ergodic control problem for the operator A as a convexoptimization subject to an elliptic equation for measures.

Then derive the Hamilton–Jacobi–Bellman (HJB) equation and establish verificationof optimality results via analytical methods, without assuming that the martingaleproblem for A is well posed.

The controlled operator

Azu(x , z) :=∑i,j

aij(x)∂2u

∂xi∂xj(x) +

bi (x , z)∂u

∂xi(x)

(u(x + y)− u(x)− 1{|y|≤1}〈y ,∇u(x)〉

)νx(dy) .

z is a control parameter that lives in a compact metric space Z.

νx(dy) is a finite Borel measure on Rd for each x .

x 7→ νx(A) is a Borel measurable function for each Borel set A.

d ≥ 2.

The coefficients of A are assumed to satisfy the following.

Assumption

(a) The matrix a = [aij ] is symmetric, positive definite, and locally Lipschitz continuous.The drift b : Rd ×Z → Rd is continuous.

(b) The map x 7→ ν(x) := νx(Rd) is locally bounded.

(c) the map x 7→ νx(K − x) is bounded on Rd for any fixed compact set K ⊂ Rd .

Azu(x , z) :=∑i,j

aij(x)∂2u

∂xi∂xj(x) +

bi (x , z)∂u

∂xi(x)

(u(x + y)− u(x)− 1{|y|≤1}〈y ,∇u(x)〉

)νx(dy) .

d ≥ 2.

Assumption

Azu(x , z) :=∑i,j

aij(x)∂2u

∂xi∂xj(x) +

bi (x , z)∂u

∂xi(x)

(u(x + y)− u(x)− 1{|y|≤1}〈y ,∇u(x)〉

)νx(dy) .

d ≥ 2.

Assumption

Some notation

Let B(Rd ,Z) denote the set of Borel measurable maps v : Rd → Z. Such a map vis called a stationary Markov control, and we use the symbol Vsm to denote thisclass of controls.

For v ∈ Vsm, we use the simplified notation bv (x) := b(x , v(x)

), and define Av , Rv

and %v analogously.

Relaxed stationary Markov control: v ∈ Vsm may be viewed as a kernel onP(Z)× Rd

bv (x) :=

∫Zb(x , z) v(dz |x) ,

and analogously for Rv .

Past work

The only treatment for the ergodic control problem with non-local operators in Rd

we could find is Menaldi & Robin (1997), under very strong blanket stabilityhypotheses.

Even though the Levy measure here is finite, there is no regularity assumption on thekernel, and this makes the problem quite difficult.

Infinitesimally invariant measures

We fix a countable dense subset C of C20 (Rd) consisting of functions with compact

supports.

Definition

A probability measure µv ∈ P(Rd), v ∈ Vsm, is called infinitesimal invariant under Av if

A∗vµv = 0 ⇐⇒∫Rd

Av f (x)µv (dx) = 0 ∀ f ∈ C .

If such a µv exists, then we say that v is a stable control, and define the (infinitesimal)ergodic occupation measure πv ∈ P(Rd ×Z) by πv (dx , dz) := µv (dx) v(dz | x).

Vssm : the set of stable controls.

M : the set of infinitesimal invariant probability measures.

G : the set of ergodic occupation measures.

The ergodic control problem for the operator A

Let R : Rd ×Z 7→ R+ be a continuous coercive function, which we refer to as therunning cost function.

Recall the set of infinitesimal ergodic occupation measures (a closed and convexsubset of P(Rd ×Z))

G ={π ∈ P(Rd ×Z) :

∫Rd×Z

Az f (x)π(dx , dz) = 0 for all f ∈ C}.

The ergodic control problem

%∗ := infπ∈G

π(R) = infπ∈G

∫Rd×Z

Rdπ .

For v ∈ Vssm, we let %v := πv (R), and we say that v is optimal if %v = %∗.

The ergodic control problem for the operator A

Let R : Rd ×Z 7→ R+ be a continuous coercive function, which we refer to as therunning cost function.

Recall the set of infinitesimal ergodic occupation measures (a closed and convexsubset of P(Rd ×Z))

G ={π ∈ P(Rd ×Z) :

∫Rd×Z

Az f (x)π(dx , dz) = 0 for all f ∈ C}.

The ergodic control problem

%∗ := infπ∈G

π(R) = infπ∈G

∫Rd×Z

R dπ .

For v ∈ Vssm, we let %v := πv (R), and we say that v is optimal if %v = %∗.

Structural Hypotheses

Note that there was no assumption on linear growth of coefficients to prevent finitetime explosion of solutions.

Two Hypotheses

(H1) For any v ∈ Vsm, the equation Avu − u = 0 has no bounded positive solutionu ∈W

2,dloc(Rd).

(H2) There exist v ∈ Vsm, a nonnegative V ∈ C2(Rd), an open ball B, and a positiveconstant κ0 such that

AvV(x) ≤ κ01B(x)− Rv (x) ∀ x ∈ Rd .

Structural Hypotheses

Note that there was no assumption on linear growth of coefficients to prevent finitetime explosion of solutions.

Two Hypotheses

(H1) For any v ∈ Vsm, the equation Avu − u = 0 has no bounded positive solutionu ∈W

2,dloc(Rd).

(H2) There exist v ∈ Vsm, a nonnegative V ∈ C2(Rd), an open ball B, and a positiveconstant κ0 such that

AvV(x) ≤ κ01B(x)− Rv (x) ∀ x ∈ Rd .

The α-discounted HJB

Theorem (α-discounted HJB)

For any α ∈ (0, 1), there exists a minimal nonnegative solution Vα ∈W2,ploc(Rd), p > 1, to

the HJB equationminz∈Z

[Az Vα(x) + R(x , z)

]= αVα(x) .

Moreover, infRd αVα ≤ %∗, and the infimum of Vα is attained in the set

Γ◦ :={x ∈ Rd : sup

z∈ZR(x , z) ≤ %∗

Concerning passage to the limit as α ↘ 0

Recall the Harnack inequality: In it’s simplest form it states that if u be a non-negativeharmonic function in D (i.e., ∆u = 0), then, for any bounded subdomain D ′ b D,

supD′

u ≤ C(d ,D,D ′) infD′

Example (Harnack fails)

Consider an operator A in R2, with a the identity matrix, b = (3, 0), and ν = νx a Diracmass at x = (3, 0). Let fε ∈ C2(R2), with ε ∈ (0, 1), be defined in polar coordinates by

fε(r , θ) := − log(r)1{r≥ε} +(

34− r2

ε2 + r4

4ε4 − log(ε))1{r<ε} .

Let uε be a function taking valuesfε(r , θ) , on B1 ,(

4ε2 − 4r2

ε4 + fε(r , θ))1{r<ε} , on B1(x) ,

nonnegative , o.w.

Then uε is nonnegative on R2 and satisfies Auε = 0 in B1.However, uε(0,θ)

uε(e−1,θ)= − log(ε).

Concerning passage to the limit as α ↘ 0

Recall the Harnack inequality: In it’s simplest form it states that if u be a non-negativeharmonic function in D (i.e., ∆u = 0), then, for any bounded subdomain D ′ b D,

supD′

u ≤ C(d ,D,D ′) infD′

Example (Harnack fails)

Consider an operator A in R2, with a the identity matrix, b = (3, 0), and ν = νx a Diracmass at x = (3, 0). Let fε ∈ C2(R2), with ε ∈ (0, 1), be defined in polar coordinates by

fε(r , θ) := − log(r)1{r≥ε} +(

34− r2

ε2 + r4

4ε4 − log(ε))1{r<ε} .

Let uε be a function taking valuesfε(r , θ) , on B1 ,(

4ε2 − 4r2

ε4 + fε(r , θ))1{r<ε} , on B1(x) ,

nonnegative , o.w.

Then uε is nonnegative on R2 and satisfies Auε = 0 in B1.However, uε(0,θ)

uε(e−1,θ)= − log(ε).

The ergodic HJB equation

Theorem (ergodic HJB)

Let Vα, α ∈ (0, 1), be the family of solutions in α–discounted HJB equations. Then, asα↘ 0, Vα − Vα(0) converges in C1,r (BR) for any r ∈ (0, 1) and R > 0, to a functionV ∈W

2,ploc(Rd) for any p > 1, which is bounded from below in Rd and solves

minz∈Z

[Az V (x) + R(x , z)

]= % , (1)

with % = %∗. Also αVα(x)→ %∗ uniformly on compact sets. In addition, the solution of(1) with % = %∗ is unique in the class of functions V ∈W

2,dloc(Rd), satisfying V (0) = 0,

which are bounded from below in Rd . For % < %∗, there is no such solution.

Theorem (verification)

If v ∈ Vssm is optimal, then it satisfies

biv (x) ∂iV (x) + Rv (x) = inf

[bi (x , z)∂iV (x) + R(x , z)

]a.e. x ∈ Rd . (2)

In addition, provided V is inf-compact, any stable v ∈ Vssm which satisfies (2) isnecessarily optimal.

The ergodic HJB equation

Theorem (ergodic HJB)

Let Vα, α ∈ (0, 1), be the family of solutions in α–discounted HJB equations. Then, asα↘ 0, Vα − Vα(0) converges in C1,r (BR) for any r ∈ (0, 1) and R > 0, to a functionV ∈W

2,ploc(Rd) for any p > 1, which is bounded from below in Rd and solves

minz∈Z

[Az V (x) + R(x , z)

]= % , (1)

with % = %∗. Also αVα(x)→ %∗ uniformly on compact sets. In addition, the solution of(1) with % = %∗ is unique in the class of functions V ∈W

2,dloc(Rd), satisfying V (0) = 0,

which are bounded from below in Rd . For % < %∗, there is no such solution.

Theorem (verification)

If v ∈ Vssm is optimal, then it satisfies

biv (x) ∂iV (x) + Rv (x) = inf

[bi (x , z)∂iV (x) + R(x , z)

]a.e. x ∈ Rd . (2)

In addition, provided V is inf-compact, any stable v ∈ Vssm which satisfies (2) isnecessarily optimal.

More regularity if we impose some structure on ν

Theorem (the Poisson equation)

We assume (H1) and one of the following:

(a) ν = νx is translation invariant and has compact support.

(b) νx has locally compact support and a density ψx ∈ Lp(Rd) for some p > d2

, suchthat x 7→ ‖ψx‖Lp(Rd ) is locally bounded

Let v ∈ Vssm be such that Rv is coercive relative to %v . Then, up to an additive constant,there exists a unique V ∈W

2,dloc(Rd) which is bounded from below in Rd , and satisfies

Av V (x) + Rv (x) = β ∀ x ∈ Rd ,

for some β = %v . For β < %v , there is no such solution.

Theorem (stronger results)

Grant the hypotheses of the preceding theorem. Then the results on the ergodic HJBhold without assuming (H2). Moreover, provided V is inf-compact, a control v ∈ Vsm isoptimal if and only if it it a selector from the minimizer of the HJB.

More regularity if we impose some structure on ν

Theorem (the Poisson equation)

We assume (H1) and one of the following:

(a) ν = νx is translation invariant and has compact support.

(b) νx has locally compact support and a density ψx ∈ Lp(Rd) for some p > d2

, suchthat x 7→ ‖ψx‖Lp(Rd ) is locally bounded

Let v ∈ Vssm be such that Rv is coercive relative to %v . Then, up to an additive constant,there exists a unique V ∈W

2,dloc(Rd) which is bounded from below in Rd , and satisfies

Av V (x) + Rv (x) = β ∀ x ∈ Rd ,

for some β = %v . For β < %v , there is no such solution.

Theorem (stronger results)

Grant the hypotheses of the preceding theorem. Then the results on the ergodic HJBhold without assuming (H2). Moreover, provided V is inf-compact, a control v ∈ Vsm isoptimal if and only if it it a selector from the minimizer of the HJB.

Thank you!

ergodic properties of lévy-driven sdes arising from ... · the hal n-whitt regime," ann....

Documents