The Multiple-Urn Ehrenfest ModelA Look into Eigenanalysis and Hitting Times
Jacquelyn Combellick, Katie Patterson, Daniel RabanAdvised by Yung-Pin Chen
Willamette Mathematics Consortium REU
July 30, 2016
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 1 / 42
What will we cover today?
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 2 / 42
What will we cover today?
(1) Markov chains
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 2 / 42
What will we cover today?
(1) Markov chains
(2) The Ehrenfest urn model
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 2 / 42
What will we cover today?
(1) Markov chains
(2) The Ehrenfest urn model
(3) Eigenanalysis
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 2 / 42
What will we cover today?
(1) Markov chains
(2) The Ehrenfest urn model
(3) Eigenanalysis
(4) Mixing times
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 2 / 42
What will we cover today?
(1) Markov chains
(2) The Ehrenfest urn model
(3) Eigenanalysis
(4) Mixing times
(5) Hitting times
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 2 / 42
What will we cover today?
(1) Markov chains
(2) The Ehrenfest urn model
(3) Eigenanalysis
(4) Mixing times
(5) Hitting times
(6) Summary
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 2 / 42
The Ehrenfest urn model example
Urn 0 Urn 1
1 4
3
M 2
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 3 / 42
What is a Markov chain?
A Markov chain is a sequence of random variables X0,X1,X2, ... such thatPr[Xn+1 = sn+1 |X0 = s0,X1 = s1, ...,Xn = sn]=Pr[Xn+1 = sn+1 |Xn=sn]where s0, s1, ..., sn, sn+1 are elements in the state space of the Markovchain.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 4 / 42
What is a Markov chain?
A Markov chain is a sequence of random variables X0,X1,X2, ... such thatPr[Xn+1 = sn+1 |X0 = s0,X1 = s1, ...,Xn = sn]=Pr[Xn+1 = sn+1 |Xn=sn]where s0, s1, ..., sn, sn+1 are elements in the state space of the Markovchain.
This is known as the memoryless property.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 4 / 42
What is a Markov chain?
A Markov chain is a sequence of random variables X0,X1,X2, ... such thatPr[Xn+1 = sn+1 |X0 = s0,X1 = s1, ...,Xn = sn]=Pr[Xn+1 = sn+1 |Xn=sn]where s0, s1, ..., sn, sn+1 are elements in the state space of the Markovchain.
This is known as the memoryless property.
Let Xn denote the number of balls in Urn 1 at time n = 0, 1, 2.... Then(X0,X1, ...) forms a Markov chain with the state space S = {0, 1, ...,M}.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 4 / 42
The Ehrenfest urn model example
Urn 0 Urn 1
1 4
3
M 2
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 5 / 42
Transition matrix
A k × k matrix P is said to be a transition matrix of a Markov chain(X0,X1, ...) with state space S = {s1, ..., sk} if
Pr[Xn+1 = sj | Xn = si ] = Pi ,j .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 6 / 42
Transition matrix
A k × k matrix P is said to be a transition matrix of a Markov chain(X0,X1, ...) with state space S = {s1, ..., sk} if
Pr[Xn+1 = sj | Xn = si ] = Pi ,j .
For the Ehrenfest urn model with M=5 balls, the transition matrix is:
0 1 2 3 4 5
0 0 1 0 0 0 01 1/5 0 4/5 0 0 02 0 2/5 0 3/5 0 03 0 0 3/5 0 2/5 04 0 0 0 4/5 0 1/55 0 0 0 0 1 0
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 6 / 42
Stationary distribution
Let (X0,X1, ...) be a Markov chain with state space S = {s1, ..., sk} andtransition matrix P. A row vector π=(π1, ..., πk ) is said to be a stationary
distribution for the Markov chain, if it satisfies
(i)πi ≥ 0 for i = 1, ..., k, and∑k
i=1 πi = 1, and
(ii)πP=π, meaning that∑k
i=1 πiPi ,j = πj for j = 1, ..., k.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 7 / 42
The 3-urn Ehrenfest model
Urn 0 Urn 1 Urn 2
1 4
3
M 2
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 8 / 42
The d -urn Ehrenfest model
Urn 0 Urn 1 Urn d-1
1
M-1
3
M 2
. . .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 9 / 42
Motivation for studying the multiple-urn Ehrenfest model
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 10 / 42
Motivation for studying the multiple-urn Ehrenfest model
Real-world applications:
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 10 / 42
Motivation for studying the multiple-urn Ehrenfest model
Real-world applications:
(1) Population migration
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 10 / 42
Motivation for studying the multiple-urn Ehrenfest model
Real-world applications:
(1) Population migration(expected hitting time)
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 10 / 42
Motivation for studying the multiple-urn Ehrenfest model
Real-world applications:
(1) Population migration(expected hitting time)
(2) Statistical mechanics, thermodynamics, and diffusion
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 10 / 42
Motivation for studying the multiple-urn Ehrenfest model
Real-world applications:
(1) Population migration(expected hitting time)
(2) Statistical mechanics, thermodynamics, and diffusion(stationary distribution)
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 10 / 42
Motivation for studying the multiple-urn Ehrenfest model
Real-world applications:
(1) Population migration(expected hitting time)
(2) Statistical mechanics, thermodynamics, and diffusion(stationary distribution)
(3) Treatment allocation
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 10 / 42
Why is eigenanalysis important?
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 11 / 42
Why is eigenanalysis important?
(1) Eigenvalues can tell how a Markov chain behaves.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 11 / 42
Why is eigenanalysis important?
(1) Eigenvalues can tell how a Markov chain behaves.
For example, a finite Markov chain converges to a stationarydistribution if and only if its transition matrix has eigenvalue 1 withmultiplicity 1 and all other eigenvalues are of modulus less than 1.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 11 / 42
Why is eigenanalysis important?
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 12 / 42
Why is eigenanalysis important?
(2) We can diagonalize a transition matrix P and compute Pn easily,
provided P is diagonalizable.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 12 / 42
Why is eigenanalysis important?
(2) We can diagonalize a transition matrix P and compute Pn easily,
provided P is diagonalizable.
That is, we can find an invertible matrix A such that
P = A−1
D A,
where D is a diagonal matrix with all the eigenvalues of P on its maindiagonal.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 12 / 42
Why is eigenanalysis important?
(2) We can diagonalize a transition matrix P and compute Pn easily,
provided P is diagonalizable.
That is, we can find an invertible matrix A such that
P = A−1
D A,
where D is a diagonal matrix with all the eigenvalues of P on its maindiagonal.
This helps us compute Pn:
Pn = A
−1D
nA.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 12 / 42
Eigenvalues of the 3-urn model
For the 3-urn model, we started by observing the eigenvalues λ for variousvalues of M, the number of balls in the model.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 13 / 42
Eigenvalues of the 3-urn model
For the 3-urn model, we started by observing the eigenvalues λ for variousvalues of M, the number of balls in the model.
M = 1: λ = 1, −12 , −1
2 .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 13 / 42
Eigenvalues of the 3-urn model
For the 3-urn model, we started by observing the eigenvalues λ for variousvalues of M, the number of balls in the model.
M = 1: λ = 1, −12 , −1
2 .
M = 2: λ = 1, 14 , 1
4 , −12 , −1
2 , −12 .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 13 / 42
Eigenvalues of the 3-urn model
For the 3-urn model, we started by observing the eigenvalues λ for variousvalues of M, the number of balls in the model.
M = 1: λ = 1, −12 , −1
2 .
M = 2: λ = 1, 14 , 1
4 , −12 , −1
2 , −12 .
M = 3: λ = 1, 12 , 1
2 , 0, 0, 0, −12 , −1
2 , −12 , −1
2 .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 13 / 42
Eigenvalues of the 3-urn model
For the 3-urn model, we started by observing the eigenvalues λ for variousvalues of M, the number of balls in the model.
M = 1: λ = 1, −12 , −1
2 .
M = 2: λ = 1, 14 , 1
4 , −12 , −1
2 , −12 .
M = 3: λ = 1, 12 , 1
2 , 0, 0, 0, −12 , −1
2 , −12 , −1
2 .
M = 4: λ = 1, 58 , 5
8 , 14 , 1
4 , 14 , −1
8 , −18 , −1
8 , −18 , −1
2 , −12 , −1
2 , −12 , −1
2 .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 13 / 42
Eigenvalues of the 3-urn model
For the 3-urn model, we started by observing the eigenvalues λ for variousvalues of M, the number of balls in the model.
M = 1: λ = 1, −12 , −1
2 .
M = 2: λ = 1, 14 , 1
4 , −12 , −1
2 , −12 .
M = 3: λ = 1, 12 , 1
2 , 0, 0, 0, −12 , −1
2 , −12 , −1
2 .
M = 4: λ = 1, 58 , 5
8 , 14 , 1
4 , 14 , −1
8 , −18 , −1
8 , −18 , −1
2 , −12 , −1
2 , −12 , −1
2 .
Our observation:
For the 3-urn model with M balls, the transition matrix has (M + 1)distinct eigenvalues equally distanced between 1 and −1
2 , and the kthlargest eigenvalue has multiplicity k.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 13 / 42
Eigenvalues of the 3-urn model
For the 3-urn model, we started by observing the eigenvalues λ for variousvalues of M, the number of balls in the model.
M = 1: λ = 1, −12 , −1
2 .
M = 2: λ = 1, 14 , 1
4 , −12 , −1
2 , −12 .
M = 3: λ = 1, 12 , 1
2 , 0, 0, 0, −12 , −1
2 , −12 , −1
2 .
M = 4: λ = 1, 58 , 5
8 , 14 , 1
4 , 14 , −1
8 , −18 , −1
8 , −18 , −1
2 , −12 , −1
2 , −12 , −1
2 .
Our observation:
For the 3-urn model with M balls, the transition matrix has (M + 1)distinct eigenvalues equally distanced between 1 and −1
2 , and the kthlargest eigenvalue has multiplicity k. Let α = 3
2M . Then the eigenvaluesare given by
1, (1 − α)2, (1 − 2α)3, ...,
(
−1
2
)
M+1
.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 13 / 42
Mark Kac’s results
In 1947, mathematician Mark Kac examined the Ehrenfest urn model with2 urns and found that the eigenvectors and eigenvalues of the transitionmatrix are determined by a system of linear equations using the generatingfunction
f (z) =
∞∑
k=0
xkzk
where xk is the kth component of the eigenvector x.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 14 / 42
Mark Kac’s results
Using Kac’s function as a model, we define the following polynomial togenerate the row eigenvectors for the 3-urn model:
f M(z1, z2) =
M∑
s1=0
M−s1∑
s2=0
a(s1,s2)zs11 z s2
2
where a(s1,s2) is the (s1, s2)th component of the eigenvector a from thetransition matrix P.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 15 / 42
Transition probabilities
The possible states for the 3-urn model are given by s = (s1, s2). Thetransition probabilites are then
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 16 / 42
Transition probabilities
The possible states for the 3-urn model are given by s = (s1, s2). Thetransition probabilites are then
Pr[(s1, s2) → (s1 − 1, s2)] =s1
2M
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 16 / 42
Transition probabilities
The possible states for the 3-urn model are given by s = (s1, s2). Thetransition probabilites are then
Pr[(s1, s2) → (s1 − 1, s2)] =s1
2M
Pr[(s1, s2) → (s1 + 1, s2)] =M − s1 − s2
2M
Pr[(s1, s2) → (s1 + 1, s2 − 1)] =s2
2M
Pr[(s1, s2) → (s1 − 1, s2 + 1)] =s1
2M
Pr[(s1, s2) → (s1, s2 + 1)] =M − s1 − s2
2M
Pr[(s1, s2) → (s1, s2 − 1)] =s2
2M.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 16 / 42
Derived partial differential equation
Using the fact that aP = λa for any eigenvector a with eigenvalue λ, weare able to substitute in our transition probabilities, multiply each term byz s11 z s2
2 , and sum over all of s1 and s2.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 17 / 42
Derived partial differential equation
Using the fact that aP = λa for any eigenvector a with eigenvalue λ, weare able to substitute in our transition probabilities, multiply each term byz s11 z s2
2 , and sum over all of s1 and s2.
After collecting like terms and simplifying, we obtain the partial differentialequation
(1 − z1)(1 + z1 + z2)∂f M
∂z1+ (1 − z2)(1 + z1 + z2)
∂f M
∂z2
+ M(z1 + z2 − 2λ)f M(z1, z2) = 0.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 17 / 42
Generating function and eigenvalues for d = 3 urns
Theorem
The coefficients of the functions
f M(r1,r2)
(z1, z2) = (1 − z1)r1(1 − z2)
r2(1 + z1 + z2)r3 ,
where r1, r2, and r3 are nonnegative integers and r1 + r2 + r3 = M, define a
set of linearly independent eigenvectors of the 3-urn M-ball transition
matrix.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 18 / 42
Generating function and eigenvalues for d = 3 urns
Theorem
The coefficients of the functions
f M(r1,r2)
(z1, z2) = (1 − z1)r1(1 − z2)
r2(1 + z1 + z2)r3 ,
where r1, r2, and r3 are nonnegative integers and r1 + r2 + r3 = M, define a
set of linearly independent eigenvectors of the 3-urn M-ball transition
matrix.
Corollary
The eigenvalues of the 3-urn M-ball transition matrix are λ = 32M r3 −
12
for r3 ∈ {0, · · · ,M}. Eigenvalue λ has multiplicity 2M3 (1 − λ) + 1.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 18 / 42
Example: The row eigenvector matrix A for M = 3 ballsConsider the function f 3
(r1,r2)(z1, z2) = (1 − z1)
r1(1 − z2)r2(1 + z1 + z2)
r3 .
For r1 + r2 = 2M3 (1 − λ), we have:
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 19 / 42
Example: The row eigenvector matrix A for M = 3 ballsConsider the function f 3
(r1,r2)(z1, z2) = (1 − z1)
r1(1 − z2)r2(1 + z1 + z2)
r3 .
For r1 + r2 = 2M3 (1 − λ), we have:
0 = 0 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 19 / 42
Example: The row eigenvector matrix A for M = 3 ballsConsider the function f 3
(r1,r2)(z1, z2) = (1 − z1)
r1(1 − z2)r2(1 + z1 + z2)
r3 .
For r1 + r2 = 2M3 (1 − λ), we have:
0 = 0 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1
1 = 1 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1
2
1 = 0 + 1 = r1 + r2 = 2M3 (1 − λ), λ = 1
2
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 19 / 42
Example: The row eigenvector matrix A for M = 3 ballsConsider the function f 3
(r1,r2)(z1, z2) = (1 − z1)
r1(1 − z2)r2(1 + z1 + z2)
r3 .
For r1 + r2 = 2M3 (1 − λ), we have:
0 = 0 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1
1 = 1 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1
2
1 = 0 + 1 = r1 + r2 = 2M3 (1 − λ), λ = 1
2
2 = 2 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 0
2 = 1 + 1 = r1 + r2 = 2M3 (1 − λ), λ = 0
2 = 0 + 2 = r1 + r2 = 2M3 (1 − λ), λ = 0
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 19 / 42
Example: The row eigenvector matrix A for M = 3 ballsConsider the function f 3
(r1,r2)(z1, z2) = (1 − z1)
r1(1 − z2)r2(1 + z1 + z2)
r3 .
For r1 + r2 = 2M3 (1 − λ), we have:
0 = 0 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1
1 = 1 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 1
2
1 = 0 + 1 = r1 + r2 = 2M3 (1 − λ), λ = 1
2
2 = 2 + 0 = r1 + r2 = 2M3 (1 − λ), λ = 0
2 = 1 + 1 = r1 + r2 = 2M3 (1 − λ), λ = 0
2 = 0 + 2 = r1 + r2 = 2M3 (1 − λ), λ = 0
3 = 3 + 0 = r1 + r2 = 2M3 (1 − λ), λ = −1
2
3 = 2 + 1 = r1 + r2 = 2M3 (1 − λ), λ = −1
2
3 = 1 + 2 = r1 + r2 = 2M3 (1 − λ), λ = −1
2
3 = 0 + 3 = r1 + r2 = 2M3 (1 − λ), λ = −1
2
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 19 / 42
Example: The row eigenvector matrix A for M = 3 ballsWith these values for (r1, r2), we can assess f 3
(r1,r2)(z1, z2):
f 3(0,0)(z1, z2) = (1 + z1 + z2)
3 λ = 1
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 20 / 42
Example: The row eigenvector matrix A for M = 3 ballsWith these values for (r1, r2), we can assess f 3
(r1,r2)(z1, z2):
f 3(0,0)(z1, z2) = (1 + z1 + z2)
3 λ = 1
f 3(1,0)(z1, z2) = (1 − z1)
1 (1 + z1 + z2)2 λ = 1
2
f 3(0,1)(z1, z2) = (1 − z2)
1(1 + z1 + z2)2
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 20 / 42
Example: The row eigenvector matrix A for M = 3 ballsWith these values for (r1, r2), we can assess f 3
(r1,r2)(z1, z2):
f 3(0,0)(z1, z2) = (1 + z1 + z2)
3 λ = 1
f 3(1,0)(z1, z2) = (1 − z1)
1 (1 + z1 + z2)2 λ = 1
2
f 3(0,1)(z1, z2) = (1 − z2)
1(1 + z1 + z2)2
f 3(2,0)(z1, z2) = (1 − z1)
2 (1 + z1 + z2)1 λ = 0
f 3(1,1)(z1, z2) = (1 − z1)
1(1 − z2)1(1 + z1 + z2)
1
f 3(0,2)(z1, z2) = (1 − z2)
2(1 + z1 + z2)1
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 20 / 42
Example: The row eigenvector matrix A for M = 3 ballsWith these values for (r1, r2), we can assess f 3
(r1,r2)(z1, z2):
f 3(0,0)(z1, z2) = (1 + z1 + z2)
3 λ = 1
f 3(1,0)(z1, z2) = (1 − z1)
1 (1 + z1 + z2)2 λ = 1
2
f 3(0,1)(z1, z2) = (1 − z2)
1(1 + z1 + z2)2
f 3(2,0)(z1, z2) = (1 − z1)
2 (1 + z1 + z2)1 λ = 0
f 3(1,1)(z1, z2) = (1 − z1)
1(1 − z2)1(1 + z1 + z2)
1
f 3(0,2)(z1, z2) = (1 − z2)
2(1 + z1 + z2)1
f 3(3,0)(z1, z2) = (1 − z1)
3 λ = −12
f 3(2,1)(z1, z2) = (1 − z1)
2(1 − z2)1
f 3(1,2)(z1, z2) = (1 − z1)
1(1 − z2)2
f 3(0,3)(z1, z2) = (1 − z2)
3 .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 20 / 42
Example: The row eigenvector matrix A for M = 3 ballsExpanding the two polynomials for λ = 1
2 gives us the coefficients andtherefore the components of the corresponding eigenvectors:
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 21 / 42
Example: The row eigenvector matrix A for M = 3 ballsExpanding the two polynomials for λ = 1
2 gives us the coefficients andtherefore the components of the corresponding eigenvectors:
f 3(1,0)(z1, z2) = (1 − z1)(1 + z1 + z2)
2
= 1 + z1 + 2z2 − z21 + 0z1z2 + z2
2 − z31 − 2z2
1z2 − z1z22 + 0z3
2
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 21 / 42
Example: The row eigenvector matrix A for M = 3 ballsExpanding the two polynomials for λ = 1
2 gives us the coefficients andtherefore the components of the corresponding eigenvectors:
f 3(1,0)(z1, z2) = (1 − z1)(1 + z1 + z2)
2
= 1 + z1 + 2z2 − z21 + 0z1z2 + z2
2 − z31 − 2z2
1z2 − z1z22 + 0z3
2
and
f 3(0,1)(z1, z2) = (1 − z2)(1 + z1 + z2)
2
= 1 + 2z1 + z2 + z21 + 0z1z2 − z2
2 + 0z31 − z2
1z2 − 2z1z22 − z3
2 .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 21 / 42
Example: The row eigenvector matrix A for M = 3 ballsExpanding the two polynomials for λ = 1
2 gives us the coefficients andtherefore the components of the corresponding eigenvectors:
f 3(1,0)(z1, z2) = (1 − z1)(1 + z1 + z2)
2
= 1 + z1 + 2z2 − z21 + 0z1z2 + z2
2 − z31 − 2z2
1z2 − z1z22 + 0z3
2
and
f 3(0,1)(z1, z2) = (1 − z2)(1 + z1 + z2)
2
= 1 + 2z1 + z2 + z21 + 0z1z2 − z2
2 + 0z31 − z2
1z2 − 2z1z22 − z3
2 .
The row eigenvectors corresponding to each of these polynomials are
(1, 1, 2,−1, 0, 1,−1,−2,−1, 0)
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 21 / 42
Example: The row eigenvector matrix A for M = 3 ballsExpanding the two polynomials for λ = 1
2 gives us the coefficients andtherefore the components of the corresponding eigenvectors:
f 3(1,0)(z1, z2) = (1 − z1)(1 + z1 + z2)
2
= 1 + z1 + 2z2 − z21 + 0z1z2 + z2
2 − z31 − 2z2
1z2 − z1z22 + 0z3
2
and
f 3(0,1)(z1, z2) = (1 − z2)(1 + z1 + z2)
2
= 1 + 2z1 + z2 + z21 + 0z1z2 − z2
2 + 0z31 − z2
1z2 − 2z1z22 − z3
2 .
The row eigenvectors corresponding to each of these polynomials are
(1, 1, 2,−1, 0, 1,−1,−2,−1, 0)
and(1, 2, 1, 1, 0,−1, 0,−1,−2,−1).
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 21 / 42
Example: The row eigenvector matrix A for M = 3 balls
So for f 3(r1,r2)
(z1, z2), we have the row eigenvector matrix
1 z1 z2 z21 z1z2 z2
2 z31 z2
1z2 z1z22 z3
2
(0, 0)(1, 0)(0, 1)(2, 0)(1, 1)(0, 2)(3, 0)(2, 1)(1, 2)(0, 3)
1 3 3 3 6 3 1 3 3 11 1 2 −1 0 1 −1 −2 −1 01 2 1 1 0 −1 0 −1 −2 −11 −1 1 −1 −2 0 1 1 0 01 0 0 −1 −1 −1 0 1 1 01 1 −1 0 −2 −1 0 0 1 11 −3 0 3 0 0 −1 0 0 01 −2 −1 1 2 0 0 −1 0 01 −1 −2 0 2 1 0 0 −1 01 0 −3 0 0 3 0 0 0 −1
= A .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 22 / 42
Generating function and eigenvalues for d urns
Theorem
The coefficients of the functions
f Mr (z) =
(
d−1∏
k=1
(1 − zk)rk
)(
1 +d−1∑
k=1
zk
)rd
,
where all the components of r are nonnegative integers and∑d
k=1 rk = M,
define a set of linearly independent eigenvectors of the M-ball d-urn
transition matrix.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 23 / 42
Generating function and eigenvalues for d urns
Theorem
The coefficients of the functions
f Mr (z) =
(
d−1∏
k=1
(1 − zk)rk
)(
1 +d−1∑
k=1
zk
)rd
,
where all the components of r are nonnegative integers and∑d
k=1 rk = M,
define a set of linearly independent eigenvectors of the M-ball d-urn
transition matrix.
Corollary
The eigenvalues of the d-urn M-ball transition matrix are
λ = dM(d−1) rd − 1
d−1 for rd ∈ {0, · · · ,M}. Eigenvalue λ has multiplicity(
M(d−1)d
(1−λ)+d−2d−2
)
.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 23 / 42
Finding the inverse matrix A−1 for d = 3 urns
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 24 / 42
Finding the inverse matrix A−1 for d = 3 urns
While we cannot expect there to be a pattern in the inverse of theeigenvector matrix, a remarkable pattern emerges upon inspection.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 24 / 42
Finding the inverse matrix A−1 for d = 3 urns
While we cannot expect there to be a pattern in the inverse of theeigenvector matrix, a remarkable pattern emerges upon inspection.
d = 3, M = 1:
1 1 11 −1 01 0 −1
−1
=1
3
1 1 11 −2 11 1 −2
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 24 / 42
Finding the inverse matrix A−1 for d = 3 urns
While we cannot expect there to be a pattern in the inverse of theeigenvector matrix, a remarkable pattern emerges upon inspection.
d = 3, M = 1:
1 +z1 +z2
1 −z1 +0z2
1 +0z1 −z2
1
3
1 +z1 +z2
1 −2z1 +z2
1 +z1 −2z2
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 25 / 42
Finding the inverse matrix A−1 for d = 3 urns
This pattern suggests that the rows of the inverse matrix A−1 are
generated by
f̃ M(s1,s2)
(z1, z2) =1
3M(1 − 2z1 + z2)
s1(1 + z1 − 2z2)s2(1 + z1 + z2)
s3 ,
where s1, s2, s3 are nonnegative integers such that s1 + s2 + s3 = M.s1, s2, s3 are the same as the r1, r2, r3 used to generate the correspondingrow of the original eigenvector matrix A.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 26 / 42
Finding the inverse matrix A−1 for d = 3 urns
This pattern suggests that the rows of the inverse matrix A−1 are
generated by
f̃ M(s1,s2)
(z1, z2) =1
3M(1 − 2z1 + z2)
s1(1 + z1 − 2z2)s2(1 + z1 + z2)
s3 ,
where s1, s2, s3 are nonnegative integers such that s1 + s2 + s3 = M.s1, s2, s3 are the same as the r1, r2, r3 used to generate the correspondingrow of the original eigenvector matrix A.
This function is also a valid solution to our partial differential equation.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 26 / 42
The d -urn inverse matrix
Theorem
The rows of the matrix A−1 are the coefficients of
f̃ Ms (z) =
1
dM
d−1∏
k=1
(1 − (d − 1)zk +∑
i 6=k
zi)sk
(
1 +
d−1∑
k=1
zk
)M−Pd−1
k=1 sk
,
where the row corresponding to the vector s in f̃ Ms (z) is in the same
position as the row corresponding to r in f Mr (z).
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 27 / 42
Proof: f̃Ms (z) generates A
−1
To prove that f̃ Ms (z) generates A
−1, let the coefficients of f Mr (z) and
f̃ Ms (z) be a
(M)r,s and 1
dM b(M)s,t , respectively, so that 1
dM B is the matrix
generated by f̃ Ms (z).
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 28 / 42
Proof: f̃Ms (z) generates A
−1
To prove that f̃ Ms (z) generates A
−1, let the coefficients of f Mr (z) and
f̃ Ms (z) be a
(M)r,s and 1
dM b(M)s,t , respectively, so that 1
dM B is the matrix
generated by f̃ Ms (z).
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 28 / 42
Proof: f̃Ms (z) generates A
−1
To prove that f̃ Ms (z) generates A
−1, let the coefficients of f Mr (z) and
f̃ Ms (z) be a
(M)r,s and 1
dM b(M)s,t , respectively, so that 1
dM B is the matrix
generated by f̃ Ms (z).
It is sufficient to prove that:
1
dM
∑
s
a(M)r,s b
(M)s,t =
{
1 if r = t,
0 otherwise.
This condition can be proved by induction on M.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 28 / 42
Computational complexity
Using diagonalization, we can compute the n-step transition matrix Pn
more efficiently. Assuming d << M,
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 29 / 42
Computational complexity
Using diagonalization, we can compute the n-step transition matrix Pn
more efficiently. Assuming d << M,
Ordinary matrix multiplication: ∼ nM3(d−1) operations
Diagonalization: ∼ nM(d−1) + 2M3(d−1) operations
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 29 / 42
Computational complexity
Using diagonalization, we can compute the n-step transition matrix Pn
more efficiently. Assuming d << M,
Ordinary matrix multiplication: ∼ nM3(d−1) operations
Diagonalization: ∼ nM(d−1) + 2M3(d−1) operations
Steps
Balls
100 200 400
10 0.637 0.786 0.806
20 26.861 29.771 34.201
(Ordinary matrix multiplication)
Steps
Balls
100 200 400
10 0.215 0.229 0.234
20 9.017 9.453 9.655
(Diagonalization)
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 29 / 42
Mixing times
The total variation distance between a probability distribution p and thestationary distribution π is given by
dTV (p,π) =1
2
∑
i
|pi − πi |.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 30 / 42
Mixing times
The total variation distance between a probability distribution p and thestationary distribution π is given by
dTV (p,π) =1
2
∑
i
|pi − πi |.
The mixing time of a Markov chain is the minimum number of steps ittakes for the total variation distance to be less than a given ǫ:
tmix(ǫ) = min{n : dTV (Pn,π) ≤ ǫ}
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 30 / 42
Mixing time bounds
We can bound the mixing time of the Ehrenfest urn model as follows:
ln( 12ǫ
)( 11−λ∗
− 1) ≤ tmix(ǫ) ≤ −ln(ǫπmin)(1
1−λ∗
)
where
λ∗ = 1 − d(d−1)M
is the second-largest eigenvalue of P and
πmin = 1dM .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 31 / 42
Mixing times: example
Consider d = 3 urns, M = 20 balls, ǫ = 0.25. If we start with all the ballsin one urn, tmix = 30.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 32 / 42
Mixing times: example
Consider d = 3 urns, M = 20 balls, ǫ = 0.25. If we start with all the ballsin one urn, tmix = 30.
The bounds on the mixing time are approximately
8 ≤ tmix ≤ 312.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 32 / 42
Mixing times for the multiple-urn Ehrenfest model
Using the eigenanalysis, we can precisely quantify the mixing times for themultiple urn Ehrenfest model.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 33 / 42
Mixing times for the multiple-urn Ehrenfest model
Using the eigenanalysis, we can precisely quantify the mixing times for themultiple urn Ehrenfest model.
Beginning with all balls in an urn, and with the choice of ǫ = 0.25, we have
M tmix(ǫ) dTV (Pn,π)
5 5 0.2305
10 13 0.2177
20 30 0.239
40 70 0.2407
80 158 0.2476
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 33 / 42
Mixing times for the multiple-urn Ehrenfest model
Using the eigenanalysis, we can precisely quantify the mixing times for themultiple urn Ehrenfest model.
Beginning with all balls in an urn, and with the choice of ǫ = 0.25, we have
M tmix(ǫ) dTV (Pn,π)
5 5 0.2305
10 13 0.2177
20 30 0.239
40 70 0.2407
80 158 0.2476
When M = 80, the dimensions of the transition matrix P are(82
2
)
×(82
2
)
,or 3321 × 3321. Even equipped with the eigenanalysis, it took R morethan 5 hours to compute the total variation distance.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 33 / 42
Hitting times
For a Markov chain Xn, n ≥ 0, the hitting time (or first passage time)from state r to state s is the minimum number of steps the chain takes toreach state s for the first time when the chain initially starts at state r.The expected value of such a hitting time is denoted by Er [Ts ], where
Ts = min{n ≥ 0 : Xn = s,Xi 6= s, for all i < n}.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 34 / 42
Hitting times
We have investigated the following hitting times associated with a multipleurn Ehrenfest model.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 35 / 42
Hitting times
We have investigated the following hitting times associated with a multipleurn Ehrenfest model.
(1) How long does it take to empty a full urn?
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 35 / 42
Hitting times
We have investigated the following hitting times associated with a multipleurn Ehrenfest model.
(1) How long does it take to empty a full urn?
(2) How long does it take to fill a specific empty urn?
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 35 / 42
Hitting times
We have investigated the following hitting times associated with a multipleurn Ehrenfest model.
(1) How long does it take to empty a full urn?
(2) How long does it take to fill a specific empty urn?
(3) How long does it take to transfer all balls in a full urn to an emptyurn?
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 35 / 42
Computing expected hitting times
General method for computing expected hitting times:
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 36 / 42
Computing expected hitting times
General method for computing expected hitting times:
Er [Ts] = E[Ts |X0 = r]
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 36 / 42
Computing expected hitting times
General method for computing expected hitting times:
Er [Ts] = E[Ts |X0 = r]
= 1 +∑
k
E[Ts |X1 = k] × Pr,k
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 36 / 42
Computing expected hitting times
General method for computing expected hitting times:
Er [Ts] = E[Ts |X0 = r]
= 1 +∑
k
E[Ts |X1 = k] × Pr,k
If we label τr,s = Er [Ts], it amounts to solve the linear system
τr,s = 1 +∑
k
τk,s × Pr,k,
and the number of unknowns τr,s equals the square of the size of the statespace of the Markov chain.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 36 / 42
Emptying a full urn, our method
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 37 / 42
Emptying a full urn, our method
Given a specific urn containing M − k balls, let Tk be the first time ittakes for the urn to have M − k − 1 balls.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 37 / 42
Emptying a full urn, our method
Given a specific urn containing M − k balls, let Tk be the first time ittakes for the urn to have M − k − 1 balls.
Tk =
1 with probability M−kM
,
1 + T ′k with probability k(d−2)
M(d−1) ,
1 + Tk−1 + T ′k with probability k
M(d−1) ,
where T ′k is a random variable with the same distribution as Tk .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 37 / 42
Emptying a full urn
We find that
E[Tk ] =1
(
M−1k
)
k∑
j=0
(
Mj
)
(d − 1)k−j
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 38 / 42
Emptying a full urn
We find that
E[Tk ] =1
(
M−1k
)
k∑
j=0
(
Mj
)
(d − 1)k−j= M
∫ 1
0xM−k−1
(
d − x
d − 1
)k
dx ,
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 38 / 42
Emptying a full urn
We find that
E[Tk ] =1
(
M−1k
)
k∑
j=0
(
Mj
)
(d − 1)k−j= M
∫ 1
0xM−k−1
(
d − x
d − 1
)k
dx ,
which gives us
Efull [Tempty ] =
M−1∑
k=0
E[Tk ]
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 38 / 42
Emptying a full urn
We find that
E[Tk ] =1
(
M−1k
)
k∑
j=0
(
Mj
)
(d − 1)k−j= M
∫ 1
0xM−k−1
(
d − x
d − 1
)k
dx ,
which gives us
Efull [Tempty ] =
M−1∑
k=0
E[Tk ] = M
M−1∑
k=0
(
dd−1
)k
k + 1.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 38 / 42
Filling an empty urn
Redefining Tk as the time to have k + 1 balls in an urn initially containingk balls,
E[Tk ] =d − 1(
M−1k
)
k∑
j=0
(
M
j
)
(d − 1)k−j .
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 39 / 42
Filling an empty urn
Redefining Tk as the time to have k + 1 balls in an urn initially containingk balls,
E[Tk ] =d − 1(
M−1k
)
k∑
j=0
(
M
j
)
(d − 1)k−j .
Using the same method, we find that the expected time to fill a specificempty urn is
Eempty [Tfull ] = M(d − 1)M−1∑
k=0
dk
k + 1.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 39 / 42
Filling an empty urn
Redefining Tk as the time to have k + 1 balls in an urn initially containingk balls,
E[Tk ] =d − 1(
M−1k
)
k∑
j=0
(
M
j
)
(d − 1)k−j .
Using the same method, we find that the expected time to fill a specificempty urn is
Eempty [Tfull ] = M(d − 1)M−1∑
k=0
dk
k + 1.
Starting with a full urn, the time to fill any of the other urns is
Efull1 [Tfullany] = M
M−1∑
k=0
dk
k + 1.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 39 / 42
Hitting time graphical representation
blue: time to empty a specific urnorange: time to fill a specific urn
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 40 / 42
Summary of results
(1) Developed a generating function f Mr for the eigenvectors of the
transition matrix P, first for the 3-urn model, then generalized for d
urns. f Mr also allows us to solve for the eigenvalues λ.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 41 / 42
Summary of results
(1) Developed a generating function f Mr for the eigenvectors of the
transition matrix P, first for the 3-urn model, then generalized for d
urns. f Mr also allows us to solve for the eigenvalues λ.
(2) Proved that f̃ Ms generates the inverse of the eigenvector matrix A,
both for the 3-urn model and a generalized d-urn model.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 41 / 42
Summary of results
(1) Developed a generating function f Mr for the eigenvectors of the
transition matrix P, first for the 3-urn model, then generalized for d
urns. f Mr also allows us to solve for the eigenvalues λ.
(2) Proved that f̃ Ms generates the inverse of the eigenvector matrix A,
both for the 3-urn model and a generalized d-urn model.
(3) Explored an application of the eigenanalysis involving bounding themixing time of the Ehrenfest urn model.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 41 / 42
Summary of results
(1) Developed a generating function f Mr for the eigenvectors of the
transition matrix P, first for the 3-urn model, then generalized for d
urns. f Mr also allows us to solve for the eigenvalues λ.
(2) Proved that f̃ Ms generates the inverse of the eigenvector matrix A,
both for the 3-urn model and a generalized d-urn model.
(3) Explored an application of the eigenanalysis involving bounding themixing time of the Ehrenfest urn model.
(4) Solved for general formulae for hitting times for various scenarios ofthe Ehrenfest urn model.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 41 / 42
Works cited
Blom, Gunnar. ”Mean Transition Times for the Ehrenfest urn Model.”Advances in Applied Probability. 21.2 (1989): 479-480. Web.
Blum, Avrim, John Hopcroft, and Ravindran Kannan. ”Chapter 5:Random Walks and Markov Chains.” Foundations of Data Science.
Unpublished.Grinstead, Charles M., and J. Laurie Snell. ”Chapter 11: Markov Chains.”
Introduction to Probability. Unpublished.Haggstrom, Olle. Finite Markov Chains and Algorithmic Applications.
New York: Cambridge UP, 2003.Kac, Mark. Probability and Related Topics in Physical Sciences. New
York: Interscience Publishers Inc, 1959. Print.Karlin, Samuel, and James McGregor. ”Ehrenfest Urn Models.” Journal of
Applied Probability 2.2 (1965): 352-376. Web.Levin, Davlid A., Yuval Peres, and Elizabeth L. Wilmer. Markov Chains
and Mixing Times. Unpublished.
Combellick, Patterson, Raban, Chen The Multiple-Urn Ehrenfest Model 42 / 42