the cutoff phenomenon in diffusion processes

A short introduction to the cutoff phenomenon indiffusion processes modeled by Markov chains

Carlo Lancia

Mathematical Engineering, University of Rome Tor Vergata

Rome, January 20, 2009

Carlo Lancia Short Introduction to the Cutoff Phenomenon

Outline

1 IntroductionMarkov Chains, Stationarity and Total Variation DistanceTop-at-Random ShuffleDefinition of Cutoff

2 Ehrenfest’s Urn ModelModel OverviewNumerical DataA Model for the Second Law of Thermodynamics

3 Random WalksCircular Random WalkLinear Random WalkBiased Linear Random Walk

Introduction Markov Chains, Stationarity and TV Distance

Some Definitions

Finite Markov Chain: a sequence of random variablesX0, X1, . . . , Xt, . . . taking values in Ω = 1, 2, . . . , nTransition Matrix: a n× n matrix Pij such that

P(Xt+1 = j|Xt = i) = Pij

Total Variation Distance: if λ, µ are two probabilitydistribution over Ω,

dTV (λ, µ) :=12

n∑i=1

|µ(i)− λ(i)| = supA⊂Ω

∣∣∣∣∣∑i∈A

[µ(i)− λ(i)]

∣∣∣∣∣Stationary Distribution π: probability distribution of thestationary state, π = π P

Mixing Time τ(ε): first time t such that dTV (P ti0 , π) ≤ ε

Total Variation Distance

Two distributions are close in total variation if and only ifthey are uniformly close on all subsets

ExampleLet’s suppose we succeeded in well-shuffling a deck of 52cards. Well-shuffling a deck means to sample uniformly atrandom one of 52! possible cards permutations.What happens if we see the bottom card of the deck, say theace of spades, A♠?

We loose uniformityNow the deck is arranged according to the uniform distribution overthe 51! possible permutations of 52 cards, last of them being A♠

Pay attention when dealing with total variation distanceThe total variation distance of the new deck distribution from theuniform one is (

151!− 1

)51! = 1− 1

52≈ 0.98

Total variation distance can be very unforgiving of smalldeviations from uniformity.

Introduction Top-at-Random Shuffle

Top-at-Random Shuffle

Top in at Random is the simplest model of card shuffling:A random position of the deck is chosenTop card is inserted into the deck at that positionProcess is iterated until convergence

Single iteration of the shuffling processUntil convergence meansunitl cards are arrangedaccording to the uniformdistribution over the n!deck permutations

Top-in-at-Random Analysis

We follow the bottom card of a deck of size nThis card stays at the bottom until the first time T1 a card isinserted below it;Consider the instant T2 when a second card is inserted belowthe original bottom card. The two cards under the originalbottom card are equally likely to be in relative order low-highor high-low;At time Tn−1 the original bottom card comes up to the top.By an inductive argument, all (n− 1)! arrangements of thelower cards are equally likely;When the original bottom card is inserted at random, at timeTn = Tn−1 + 1, then all n! possible arrangements of the deckare equally likely.

Expectation of Tk : E[Tk] =n∑j=1

j P(Tk = j) =n

Mixing Time Estimate: E[τ ] = 1 +n−1∑k=1

E[Tk] = n log n

j P(Tk = j) =n

E[Tk] = n log n

j P(Tk = j) =n

E[Tk] = n log n

Convergence of Top at Random modelUntil time Tn−1 we may have knowledge of the exact position ofthe original bottom card, so dTV (P t, π) ≈ 1 if t < Tn−1. At timeTn cards are arranged according to the uniform distribution, sodTV (P t, π) ≈ 0 if t > Tn.

j P(Tk = j) =n

E[Tk] = n log n

A sharp evaluation of τThe chain abruptly converges to stationarity at time Tn son log n is actually discovered to be not only a mean valuebut a also very sharp evaluation of τ .

Introduction Definition of Cutoff

The Cutoff Phenomenon

We consider a family of chains X(n)t ∈ Ωn = 1, 2, . . . , n

The n-th chain has transition kernel P (n), limit measure πn andmixing time τ (n)(ε). P tn is the state distribution after t steps.

Let an, bn be two sequences such thatbnan→ 0.

Definition

The family Ωn, X(n)t , πn exhibits cutoff at time an with a cutoff

window of size bn iff

limθ→∞

lim supn→∞

‖P dan+θbnen − πn‖TV = 0

limθ→∞

lim infn→∞

‖P ban−θbncn − πn‖TV = 1

The Case of Birth-and-Death Chains

Xt is a birth-and-death chain if Pij = 0 unless |i− j| ≤ 1Xt is a δ-lazy birth-and-death chain provided Pii = δ > 0

Definition

A family of birth-and-death chains Ωn, X(n)t , πn exhibits cutoff iff

limn→∞

τ (n)(ε)τ (n)(1− ε)

= 1 ∀ 0 < ε < 1

Cutoff for Birth-and-Death Chains

whether a family of δ-lazy birth-and-death chains exhibits cutoff itdepends on the product τ (n) · t(n)

t(n)rel is the relaxation time and is equal to (1− λn)−1

λn is the largest absolute-value of all nontrivial eigenvalues ofthe transition kernel P (n) (nontrivial means different from 1)

TheoremA family of δ-lazy birth-and-death chains exhibits cutoff ifft(n)rel = o(τ (n)(1

4)) and

τ (n)(1− ε)− τ (n)(ε) ≤

√t(n)rel · τ (n)

Ehrenfest’s Urn Model Model Overview

Ehrenfest’s Urn Model

The Ehrenfest’s Urn is a very simple model for particles diffusion.We have n balls in 2 urns

A ball is randomly chosen and moved to the other urnProcess is iterated towards stationarity

Single chain step Say Xt is the number ofballs in urn 1, then Xt isan ergodic Markov chainwhich limit measure is

π(j) = 2−n(nj

Ehrenfest’s Urn Model Numerical Data

Cutoff for the Ehrenfest’s Urn Model

Evolution of the Distribution Curve

Ehrenfest’s Urn Model A Model for the Second Law of Thermodynamics

Explaining the Second Law of Thermodynamics

How can entropy increase be compatible with a systemreturning to its starting state?

Suppose entropy is measured by

I(t) =n∑j=0

P t0 logP t0

where P t0 is the distribution of the number of balls in urn 1 after tsteps, urn 1 being initially empty.

I(t) is increasing in t because the system tends to stationarity;It is equally certain that the number of balls in urn 1 willreturn to zero infinitely often as time goes on.

Ehrenfest’s Urn Model A Model for the Second Law of Thermodynamics

Explaining the Second Law of Thermodynamics

How can entropy increase be compatible with a systemreturning to its starting state?

Suppose entropy is measured by

I(t) =n∑j=0

P t0 logP t0

Return to initial position won’t be observedIt takes about n log n steps for the chain to reach maximum entropybut about 2n steps to have a reasonable chance of returning to 0.If n is large (Avogadro’s number) we will not observe suchreturns

Random Walks Circular Random Walk

Lazy Uniform Random Walk mod n

This process is clearly a 13 -lazy birth-and-death chain, Ω = Z mod n

πn ≡ 1n

τ (n) = O(n2)

t(n)rel = O(n2)

=⇒ NO CUTOFF

Random Walks Circular Random Walk

Lazy Uniform Random Walk mod n

Random Walks Linear Random Walk

Lazy Uniform Random Walk

Also this process is a 13 -lazy birth-and-death chain, Ω = 1, . . . , n

πn ≡ 1n

τ (n) = O(n2)

t(n)rel = O(n2)

=⇒ NO CUTOFF

Regularity of dTVSince π is uniform dTV (P t, π) smoothlydecreases from 1 to 0 wherever thechain starts at t = 0

Also this process is a 13 -lazy birth-and-death chain, Ω = 1, . . . , n

πn ≡ 1n

τ (n) = O(n2)

t(n)rel = O(n2)

=⇒ NO CUTOFF

Regularity of dTVSince π is uniform dTV (P t, π) smoothlydecreases from 1 to 0 wherever thechain starts at t = 0

Random Walks Biased Linear Random Walk

Biased Lazy Uniform Random Walk

What happens if we break the transition probabilities symmetry?

DriftWe introduce a drift q > 0, so the chain has abigger chance to move right

Breaking symmetry leads to cutoffBreaking symmetry concentrates π on the right wall.If the chain starts from the left we detect cutoff.

What happens if we break the transition probabilities symmetry?

DriftWe introduce a drift q > 0, so the chain has abigger chance to move right

Breaking symmetry leads to cutoffBreaking symmetry concentrates π on the right wall.If the chain starts from the left we detect cutoff.

the cutoff phenomenon in diffusion processes

Documents