traffic matrix estimation in non-stationary environments
DESCRIPTION
Traffic Matrix Estimation in Non-Stationary Environments. Presented by R. L. Cruz Department of Electrical & Computer Engineering University of California, San Diego Joint work with Antonio Nucci Nina Taft Christophe Diot NISS Affiliates Technology Day on Internet Tomography - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/1.jpg)
Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic Matrix Estimation in Non-Stationary Environments
Presented by R. L. Cruz
Department of Electrical & Computer EngineeringUniversity of California, San Diego
Joint work with
Antonio NucciNina Taft
Christophe Diot
NISS Affiliates Technology Day on Internet TomographyMarch 28, 2003
![Page 2: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/2.jpg)
page 2Advanced Technology LaboratoriesAdvanced Technology Laboratories
The Traffic Matrix Estimation Problem
• Formulated in
Y. Vardi, “Network Tomography: Estimating Source-Destination Traffic From Link Data,” JASA, March 1995, Vol. 91, No. 433, Theory & Methods
![Page 3: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/3.jpg)
page 3Advanced Technology LaboratoriesAdvanced Technology Laboratories
The Traffic Matrix Estimation Problem
ingress
egress
Xj
Xj
Yi
PoP (Point of Presence)
Y = A X
Link Measurement Vector
Routing Matrix
“Traffic Matrix”
![Page 4: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/4.jpg)
page 4Advanced Technology LaboratoriesAdvanced Technology Laboratories
The Traffic Matrix Estimation Problem
• Importance of Problem: capacity planning, routing protocol configuration, load balancing policies, failover strategies, etc.
• Difficulties in Practice– missing data – synchronization of measurements (SNMP)– Non-Stationarity (our focus here)
• long convergence time needed to obtain estimates
![Page 5: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/5.jpg)
page 5Advanced Technology LaboratoriesAdvanced Technology Laboratories
What is Non-Stationary?
•Traffic Itself is Non-Stationary
![Page 6: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/6.jpg)
page 6Advanced Technology LaboratoriesAdvanced Technology Laboratories
What is Non-Stationary?
• Also, Routing is Non-Stationary– e.g. Due to Link Failures– Essence of Our Approach
• Purposely reconfigure routing in order to help estimate traffic matrix
– More information leads to more accurate estimates
• Effectively increases rank of A• We have developed algorithms to
reconfigure the routing for this purpose (beyond the scope of this talk)
![Page 7: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/7.jpg)
page 7Advanced Technology LaboratoriesAdvanced Technology Laboratories
Outline of Remainder of Talk
• Describe the “Stationary” Method– Stationary traffic, non-stationary routing– Stationary traffic assumption is reasonable if we
always measure traffic at the same time of day (e.g. “peak period” of a work day)
• Briefly Describe the “Non-Stationary” Method– Both non-stationary traffic and non-stationary
routing– More complex but allows estimates to be obtained
much faster
![Page 8: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/8.jpg)
page 8Advanced Technology LaboratoriesAdvanced Technology Laboratories
Network and Measurement Model
• Network with L links, N nodes, P=N(N-1) OD pair flows
– K measurement intervals, 1 ≤ k ≤ K – Y(k) is the link count vector at time k: (L x 1)– A(k) is the routing matrix at time k: (L x P)– X(k) is the O-D pair traffic vector at time k: (P
x 1)• X(k) = (x1(k) , x2(k) , … xP(k))T
Y(k) = A(k) X(k)
k [1,K]
Y(k) and A(k) can be truncated to reflect missing and redundant data
![Page 9: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/9.jpg)
page 9Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic Model: Stationary Case
x i(k) x i wi(k) k [1,K]
k
)(kxi
ix
)(kwi
• X(k) is the O-D pair traffic vector at time k: (P x 1)
X(k) = (x1(k) , x2(k) , … xP(k))T
X(k) = X + W(k)
W(k) : “Traffic Fluctuation Vector• Zero mean, covariance matrix B• B = diag(X)
![Page 10: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/10.jpg)
page 10Advanced Technology LaboratoriesAdvanced Technology Laboratories
Matrix Notation
CWAXY
KkkWkAXkAkY
],1[)()()()(
)(
...
)1(
...
)(000
0...00
00)2(0
000)1(
)(
...
)1(
)(
...
)1( 1
KW
W
W
x
x
X
kA
A
A
C
kA
A
A
kY
Y
Y
P
where:
Linear system of equations:
[LK] [LK][P] [LK][KP] [KP][P]
Choose Routing Configurations such that
Rank(A) = P
![Page 11: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/11.jpg)
page 11Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic matrix Estimation-Stationary Case
• Initial Estimate: Use Psuedo-Inverse of A:
- does not require statistics of W (covariance B)• Gauss-Markov Theorem: Assume B is known
- Unbiased, minimum variance estimate- Coincides with Maximum Likelihood Estimate if W is Gaussian
Y = AX + CW
ˆ X (0) (AT A) 1 ATY
ˆ X (AT (CBCT ) 1 A) 1 AT (CBCT ) 1Y
![Page 12: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/12.jpg)
page 12Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic matrix Estimation-Stationary Case
Y = AX + CW
• Minimum Estimation Error:
(assumes B is known)
11 ))((])ˆ)(ˆ[( ACBCAXXXXE TTT
ˆ X (AT (CBCT ) 1 A) 1 AT (CBCT ) 1Y
![Page 13: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/13.jpg)
page 13Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic matrix Estimation-Stationary Case
• Recall we assume B = cov(W) satisfies B = diag(X)• Set
ˆ B (k ) diag( ˆ X (k ))
ˆ X (k1) (AT (C ˆ B (k )CT ) 1 A) 1 AT (C ˆ B (k )CT ) 1Y
• Recursion for Estimates:
![Page 14: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/14.jpg)
page 14Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic matrix Estimation-Stationary Case
• Our estimate is a solution to the equation:
ˆ X (AT (C diag( ˆ X ) CT ) 1 A) 1 AT (C diag( ˆ X ) CT ) 1Y
• Open questions for fixed point equation:- Existence of Solution?- Uniqueness?- Is solution an un-biased estimate?
![Page 15: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/15.jpg)
page 15Advanced Technology LaboratoriesAdvanced Technology Laboratories
Numerical Example-Stationary case
N=10 nodes, L=24 links and P=90 connections.
Three set of OD pairs with mean x equal to:
– 500 Mbps, 2 Gbps and 4 Gbps.
Gaussian Traffic Fluctuations:
bx 20
![Page 16: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/16.jpg)
page 16Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1 Samples/Snapshot=1
![Page 17: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/17.jpg)
page 17Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1 Samples/Snapshot=1
![Page 18: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/18.jpg)
page 18Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1 Samples/Snapshot=15
![Page 19: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/19.jpg)
page 19Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1 Samples/Snapshot=15
![Page 20: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/20.jpg)
page 20Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1.4 Samples/Snapshot=1
![Page 21: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/21.jpg)
page 21Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1.4 Samples/Snapshot=1
![Page 22: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/22.jpg)
page 22Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1.4 Samples/Snapshot=15
![Page 23: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/23.jpg)
page 23Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1.4 Samples/Snapshot=15
![Page 24: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/24.jpg)
page 24Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary and Non-Stationary traffic
20 snapshots / 4 samples per snapshot / 5 min per sample
• Stationary Approach: 20 min per day (same time) / 20 days• Non-Stationary Approach: aggregate all the samples in one window time large 400 min (7 hours)
![Page 25: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/25.jpg)
page 25Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic Model: Non-Stationary Case
)()( kTxkx ii
• Each OD pair is cyclo-stationary:
• Each OD pair is modeled as:
• Fourier series expansion:
],1[)()()( Kkkwkmkx iii
)/2sin(
)/2cos()(
)(ˆ)(ˆ2
0
Nkn
Nknkb
kbmkm
n
N
nnini
b
]2,1[],0[
],1[],0[
bb
b
NNnNk
NnNk
)(kxi )(kxi
kk
)(kwi )(kwi
)(kmi)(kmi
![Page 26: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/26.jpg)
page 26Advanced Technology LaboratoriesAdvanced Technology Laboratories
Mean estimation Results-Non Stationary case
Three set of OD pairs
where are linear independent Gaussian variables with:
)()2sin()( twfaxtx iiiiii
bix 20
3/2
3/
0
4
2
500
ii
Gbps
Gbps
Mbps
x
)(twi
![Page 27: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/27.jpg)
page 27Advanced Technology LaboratoriesAdvanced Technology Laboratories
Non Stationary case: b=1 Link Count
![Page 28: Traffic Matrix Estimation in Non-Stationary Environments](https://reader035.vdocuments.us/reader035/viewer/2022062301/5681484b550346895db55e5d/html5/thumbnails/28.jpg)
page 28Advanced Technology LaboratoriesAdvanced Technology Laboratories
Non Stationary case: b=1 Mean estimation