on the generality of binary tree-like markov chains k. spaey - b. van houdt - c. blondia performance...

On the generality of binary tree-like On the generality of binary tree-like Markov chainsMarkov chains

K. Spaey - B. Van Houdt - C. Blondia

Performance Analysis of Telecommunication Systems (PATS) Research Group

University of Antwerp - IBBT

MAM2006 - June 12-14, 2006 - Charleston, S.C.

MAM2006MAM2006 On the generality of binary tree-like On the generality of binary tree-like Markov chainsMarkov chains

22

On the generality of binary tree-like On the generality of binary tree-like Markov chainsMarkov chains

Aim of the paper:

Show that an arbitrary tree-like Markov chain can be embedded in a binary tree-like Markov chain with a special structure


33

Tree-like QBD Markov chainsTree-like QBD Markov chains

• States are grouped into sets of m states: “nodes”

• The nodes form a d-ary tree

• Transitions: from a node to itself, to its parent node, to its child nodes

• Characterized by the matrices B and F, Dk, Uk, k = 1,...,d.


44


• Key equations:

for k = 1,...,d

• Stability condition:

needs to be stochastic for all k

• Steady state probabilities:

for all J, for all k

d

1kk

1k DV)(IUBV

1kk V)(IUR

k1

k DV)(IG

kR (J)k)(J ππ

)DR(F )()(d

1kkk

ππ

1 )eR)(I(d

1kk

π


55


• Tree-like Markov chains were introduced as a special case of the tree-structured Markov chains (Bini, Latouche & Meini, Solving nonlinear matrix equations arising in tree-like stochastic processes, Linear Algebra Appl. 366, 2003)

• Any tree-structured Markov chain can be reduced to a tree-like Markov chain(Van Houdt & Blondia, Tree structured QBD Markov chains and tree-like QBD processes, Stochastic Models 19(4), 2003)

• Any tree-like Markov chain can be embedded in a binary (d=2) tree-like Markov chain


66

Constructing the binary tree-like MCConstructing the binary tree-like MC

• Tree-like MC (Xt,Nt) Xt: nodes d-ary tree

Nt: auxiliary variable

• Nodes are denoted by strings (symbols 1,...,d) J = j1 j2 ... jn-1 jn

Root node: ø

• Auxiliary variable i = 1,...,m

• Binary tree-like MC : nodes binary tree : 2D auxiliary variable

• Nodes are denoted by binary strings (symbols 0, ) starting with a “”

Root node: ø

• Auxiliary variable (0,i) corresponding to node ø (a,i) for other nodes

i = 1,...,m a = -(d-1),...,-1,0,1,...,d-1

))M,Q(N,X( ttttˆˆˆˆ

tX̂)M,Q(N ttt

ˆˆˆ


77


• Binary notation ψ of the nodes of the d-ary tree: and ψ(ø) = ø

• 1-1 correspondence between states (J,i) of (Xt,Nt) and states (ψ(J),(0,i)) of

• Every possible transition in (Xt,Nt) between (J,i) and (J’,i’) will be mimicked by a path of transitions in between (ψ(J),(0,i)) and (ψ(J’),(0,i’))

1j1j1j1j

n1-n21

n1-n1

00000000 )jjjψ(j

2

)N,X( ttˆˆ

...

ø

0

00 0 0

000 00 00 0 00 0 0

ø

1

2

3

4 31 22 211

21

11

12

13 121

111

1111112

)N,X( ttˆˆ


88


• d-ary tree: transition from a node to its k-th child (J,i) (J+k,j) with prob. (Uk)i,j

• binary tree: (ψ(J),(0,i)) (ψ(J),(k-1,j)) with prob. (U)(0,i),(k-1,j) = (Uk)i,j

(ψ(J)0,(k-2,j)) with prob. (U0)(k-1,j),(k-2,j) = 1

... (ψ(J)0...0,(1,j)) with prob. (U0)(k-2,j),(k-3,j) ... (U0)(2,j),(1,j) = 1

(ψ(J)0...00,(0,j)) = (ψ(J+k),(0,j)) with prob. (U0)(1,j),(0,j) = 1

...

ø

0

00 0 0

000 00 00 0 00 0 0

ø

1

2

3

4 31 22 211

21

11

12

13 121

111

1111112


99


• d-ary tree: transition from a child k to its parent (J+k,i) (J,j) with prob. (Dk)i,j

• binary tree: (ψ(J+k),(0,i)) = (ψ(J)0...00,(0,i)) (ψ(J)0...0,(-1,i))

with prob. (D0)(0,i),(-1,i) = i,j, = diag(D1e)

... (ψ(J),(-(k-1),i)) with prob. (D0)(-1,i),(-2,i) ... (D0)(-(k-2),i),(-(k-1),i) = 1

(ψ(J),(0,j)) with prob. (D)(-(k-1),i),(0,j) = (-1Dk)i,j

...

ø

0

00 0 0

000 00 00 0 00 0 0

ø

1

2

3

4 31 22 211

21

11

12

13 121

111

1111112


1010


• d-ary tree: transition from a node to itself root node: (ø,i) (ø,j) with prob. Fi,j

other node: (J,i) (J,j) with prob. Bi,j

• binary tree: root node: (ø,(0,i)) (ø,(0,j)) with prob. other node: (J,(0,i)) (J,(0,j)) with prob.

ji,j)(0,i),(0, FF ˆ

ji,j)(0,i),(0, BB ˆ


1111

Calculating the steady state probabilitiesCalculating the steady state probabilities

d-ary tree like MC

•

• for k = 1,...,d

• stability condition: needs to be stochastic for all k

binary tree-like MC

•

•

• stability condition:and need to be stochastic

d

1kk

1k DV)(IUBV

1kk V)(IUR

D)V(IUD)V(IUBV 1

01

0ˆˆˆˆ

1100 )V(IUR and )V(IUR

ˆˆ

k1

k DV)(IG D)V(IG 01

0 ˆ

D)V(IG 1

ˆ

All Gk stochastic G0 and G stochastic

Algorithms for calculating the steady state probabilities: Fixed point iteration (FPI) Reduction to quadratic equations (RQE) Newton’s iteration (NI)


1212


• The matrices corresponding to the constructed binary tree-like MC, e.g., U0, U, D0, D,

have a structure that is related to the matrices that correspond to

the original d-ary tree-like MC

• Example (d=4)

F ,B ˆˆ

U)V(IG ,U)V(IG ,)V(IUR ,)V(IUR ,V 1

01

011

00ˆˆˆˆˆ

0000ΔV)-(I00

00000ΔV)-(I0

000000ΔV)-(I

000V000

0000000

0000000

0000000

V

1-

1-

1-

ˆ


1313


• Fixed point iteration (FPI) iterative algorithm:

V[N] monotonically converges to V

• Applied to binary tree:

more iterations needed taking the structure of into account

identical to applying FPI to d-ary tree

d

1kk

1k DV[N])(IUB1]V[N

BV[0]

D[N])V(IUD[N])V(IUB1][NV

B[0]V1

01

0ˆˆˆˆ

ˆˆ

V̂


1414


• Reduction to quadratic equations (RQE) iterative algorithm:

• Gi[0] = 0, i = 1,...,d

• d quadratic matrix equations solve for Gi[N+1]

• Gi[N] converges to Gi, i = 1,...,d

• Applied to binary tree:• G0[0] = G[0] = 0

• slower convergence• taking the structure of the matrices into account d quadratic matrix

equations as when applying RQE to d-ary tree

01][NGU1][NG [N]GU1][NGUIBD 2iii

1i

1k

d

1ikkkkki

1][NG for solve 01][NGU1][NG 1][NGUIBD

1][NG for solve 01][NGU1][NG [N]GUIBD2

00

02

0000

ˆ

ˆ


1515


• Newton’s iteration (NI) iterative algorithm computes the matrices Gi, i=1,...,d

converges quadratically each step requires solving an equation of the form

large linear system of equations Ax=b (inefficient)

• Applied to binary tree: each step requires solving an equation of the form

≈ Sylvester equation linear system of equations

reduction to binary tree can result in computational gain ???

d

1kkk L XK XH

L X K XH K XH 2211ˆˆˆˆˆˆˆˆ

d

1kk

Tk )H(KIA

2T21

T1 HKHK I- A ,bxA ˆˆˆˆˆˆˆˆ


1616

ConclusionsConclusions

• Any tree-like Markov chain can be embedded in a binary tree-like Markov chain with a special structure

any tree-structured Markov chain can be embedded in a binary tree-like Markov chain with a special structure

• Mainly of theoretical interest: FPI and RQE algorithms applied to binary tree do not speed up

calculations of the steady state probabilities NI algorithm: currently unclear

on the generality of binary tree-like markov chains k. spaey - b. van houdt - c. blondia performance...

Documents

states j

jbinary tree

kbinary tree

arbitrary tree

mbinary tree

nodes binary tree

generality of binary

mcdary tree