optimal network locality in distributed services
DESCRIPTION
In age of cloud computing, any equipment can become server, e.g. set-top-boxes or access routers. For service providers, a challenge consists in accurately making use of these servers. We address the problem of locating a large service (or content) into these Internet edges so that the delivery to clients is efficient from a networking point of view.TRANSCRIPT
Optimal NetworkLocality in DistributedServicesGwendal SimonDepartment of Computer ScienceInstitut Telecom - Telecom Bretagne2010
Telecom Bretagne
Institut Telecom: graduate engineering schoolsTelecom Bretagne: 1200 students (200 PhD)Computer Science: 20 full-time research lecturers
2 / 27 Gwendal Simon Network Locality in Distributed Services
Credits
Funding:Thomson R&D (now Technicolor)French grant with Orange, NDS Tech. and INRIA
Co-authors:Jimmy Leblet (post-doc)Yiping Chen (PhD student)Zhe Li (PhD student)Gilles Straub (senior researcher Thomson)Di Yuan (Ass. Professor: Linkopping Univ. Sweden)
3 / 27 Gwendal Simon Network Locality in Distributed Services
Credits
Funding:Thomson R&D (now Technicolor)French grant with Orange, NDS Tech. and INRIA
Co-authors:Jimmy Leblet (post-doc)Yiping Chen (PhD student)Zhe Li (PhD student)Gilles Straub (senior researcher Thomson)Di Yuan (Ass. Professor: Linkopping Univ. Sweden)
3 / 27 Gwendal Simon Network Locality in Distributed Services
Service Delivery Network
CLOUD
end user
data-center CDN
in-network servers
set-top-boxset-top-box
set-top-box
set-top-box
4 / 27 Gwendal Simon Network Locality in Distributed Services
Service Delivery Network
CLOUD
end user
data-center
CDN
in-network servers
set-top-boxset-top-box
set-top-box
set-top-box
4 / 27 Gwendal Simon Network Locality in Distributed Services
Service Delivery Network
CLOUD
end user
data-center CDN
in-network servers
set-top-boxset-top-box
set-top-box
set-top-box
4 / 27 Gwendal Simon Network Locality in Distributed Services
Service Delivery Network
CLOUD
end user
data-center CDN
in-network servers
set-top-boxset-top-box
set-top-box
set-top-box
4 / 27 Gwendal Simon Network Locality in Distributed Services
Service Delivery Network
CLOUD
end user
data-center CDN
in-network servers
set-top-boxset-top-box
set-top-box
set-top-box
4 / 27 Gwendal Simon Network Locality in Distributed Services
Toward a Decentralized Architecture
servers’ capacities scale down1
services scale up2
=⇒ multi-servers multi-component architectures3
1J. He, A. Chaintreau and C. Diot. “A performance evaluation of scalablelive video streaming with nano data centers” Computer Networks, 2009
2J. Pujol, V. Erramilli, and P. Rodriguez, “Divide and Conquer:Partitioning Online Social Networks” Arxiv preprint arXiv:0905.4918, 2009.
3R. Baeza-Yates, A. Gionis, F. Junqueira, V. Plachouras, and L. Telloli,“On the feasibility of multi-site web search engines”, in ACM CIKM 2009
5 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Modellingand Analysis
6 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Formulation (Assumptions)
About the n servers and the k components:only one component per serverno capacity boundscomponents are uniformly accessed
About the global service architecture:client requests are routed toward the closest servercharacteristics of links between servers are known
a generic distance (cost) function dij
7 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Formulation (Assumptions)
About the n servers and the k components:only one component per serverno capacity boundscomponents are uniformly accessed
About the global service architecture:client requests are routed toward the closest servercharacteristics of links between servers are known
a generic distance (cost) function dij
7 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Formulation (Definition)
Rainbow distance for a server i :total cost to fetch all missing components
i j1
j2
j3
j4
dij1 = 2dij2
= 3dij3
= 4dij4
=5
dij1 = 2
dij4=
5d(i) = dij1 + dij4 = 7
8 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Formulation (Definition)
Rainbow distance for a server i :total cost to fetch all missing components
i j1
j2
j3
j4
dij1 = 2dij2
= 3dij3
= 4dij4
=5
dij1 = 2
dij4=
5d(i) = dij1 + dij4 = 7
8 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Formulation (Objective)
Global goal: assign components to servers
Optimization: minimize sum of rainbow distances∑
0<i≤nd(i)
Motivations:network operator: reduce cross-domain trafficservice provider: reduce overall latencyacademic: funny unknown problem
9 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Formulation (Objective)
Global goal: assign components to servers
Optimization: minimize sum of rainbow distances∑
0<i≤nd(i)
Motivations:network operator: reduce cross-domain trafficservice provider: reduce overall latencyacademic: funny unknown problem
9 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Complexity
The problem is NP-complete:closely related with domatic partition
012
3
4
5
6
7
8 9
10 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Complexity
The problem is NP-complete:closely related with domatic partition
012
3
4
5
6
7
8 9
6
5
4
10 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Complexity
The problem is NP-complete:closely related with domatic partition
012
3
4
5
6
7
8 983
0
10 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Complexity
The problem is NP-complete:closely related with domatic partition
012
3
4
5
6
7
8 91
2
7
9
10 / 27 Gwendal Simon Network Locality in Distributed Services
Problem Complexity
The problem is NP-complete:closely related with domatic partition
012
3
4
5
6
7
8 9
6
5
4
12
7
983
0
10 / 27 Gwendal Simon Network Locality in Distributed Services
Integer Programming
xic =
{1 if component c is allocated at server i0 otherwise
y cij =
{1 if i obtains component c from j0 otherwise
Minimizen∑
i=1
k∑c=1
n∑j=1
d(i , j)y cij
Subject tok∑
c=1
xic = 1, only one component per server∑j 6=i
y cij = 1− xic , a server has c or has exactly one pointer to c
y cij ≤ xjc , a server has c from another server if this latter has c
11 / 27 Gwendal Simon Network Locality in Distributed Services
Related Works
12 / 27 Gwendal Simon Network Locality in Distributed Services
Related Works
Facility Location Problem⇒ open a subset of facilities with minimal overall cost
u1c1 = 9
u2c2 = 3
u3c3 = 5
i1 i2
3 6 8 12 11 7
13 / 27 Gwendal Simon Network Locality in Distributed Services
Related Works
Facility Location Problem⇒ open a subset of facilities with minimal overall cost
u1c1 = 9
u2c2 = 3
u3c3 = 5
i1 i2
3 6 8 12 11 7
13 / 27 Gwendal Simon Network Locality in Distributed Services
Related Works
Facility Location Problem⇒ open a subset of facilities with minimal overall cost
u1c1 = 9
u2c2 = 3
u3c3 = 5
i1 i2
3 6 8
12 11 7
13 / 27 Gwendal Simon Network Locality in Distributed Services
Related Works
Facility Location Problem⇒ open a subset of facilities with minimal overall cost
u1c1 = 9
u2c2 = 3
u3c3 = 5
i1 i2
3 6 8 12 11 7
13 / 27 Gwendal Simon Network Locality in Distributed Services
Related Works
Facility Location Problemmost variants are NP-completeclose variant is k-PUFLP: a
(32k − 1
)-approx. algo4
possible transformation from our prob. to k-PUFLP
4H. C. Huang and R. Li, “A k-product uncapacitated facility locationproblem”, European Journal of Op. Res., vol. 185, no. 2, 2008.
13 / 27 Gwendal Simon Network Locality in Distributed Services
Related Works
Facility Location Problema(32k − 1
)-approx. algo
13 / 27 Gwendal Simon Network Locality in Distributed Services
Related Works
Facility Location Problema(32k − 1
)-approx. algo
Content Delivery Networksk-median problem: no multiple servers
13 / 27 Gwendal Simon Network Locality in Distributed Services
Related Works
Facility Location Problema(32k − 1
)-approx. algo
Content Delivery Networksk-median problem: no multiple servers
Nano data centers powered by set-top-boxesuniform random allocation of components to servers
13 / 27 Gwendal Simon Network Locality in Distributed Services
Our Algorithms
14 / 27 Gwendal Simon Network Locality in Distributed Services
Approximate Algorithm
For a server i :1. compute distance d̄(i) to k − 1 closest servers2. wait until every server j with smaller d̄(j) are OK3. try to optimize locally −→ optimized state4. if impossible −→ saved state5. uncolored saved nodes get furthest components
15 / 27 Gwendal Simon Network Locality in Distributed Services
Approximate Algorithm
1
2
3
4
5
67
8
9
10
11
12
13
14
15
16
17
18
2
1
4
5
optimized2
1
4
5
3816
optimized
2
1
4
5
3816
conflict
saved but colored
2
1
4
5
3816
11
12
13
9
15
7
conflict
saved and uncolored
2
1
4
5
3816
11
12
13
9
15
10
67
colored by node 10
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
only node uncolored
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
18
choose farthest component
sorted list nearest neighbors2 1,4,53 1,8,161 2,3,168 3,11,125 1,2,411 8,12,134 2,5,716 1,3,512 8,9,1115 1,10,1110 2,6,1514 3,16,1717 5,14,1613 11,12,157 2,4,66 2,7,109 8,12,1418 4,5,17
16 / 27 Gwendal Simon Network Locality in Distributed Services
Approximate Algorithm
1
2
3
4
5
67
8
9
10
11
12
13
14
15
16
17
18
2
1
4
5
optimized
2
1
4
5
3816
optimized
2
1
4
5
3816
conflict
saved but colored
2
1
4
5
3816
11
12
13
9
15
7
conflict
saved and uncolored
2
1
4
5
3816
11
12
13
9
15
10
67
colored by node 10
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
only node uncolored
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
18
choose farthest component
sorted list nearest neighbors2 1,4,53 1,8,161 2,3,168 3,11,125 1,2,411 8,12,134 2,5,716 1,3,512 8,9,1115 1,10,1110 2,6,1514 3,16,1717 5,14,1613 11,12,157 2,4,66 2,7,109 8,12,1418 4,5,17
16 / 27 Gwendal Simon Network Locality in Distributed Services
Approximate Algorithm
1
2
3
4
5
67
8
9
10
11
12
13
14
15
16
17
18
2
1
4
5
optimized
2
1
4
5
3816
optimized
2
1
4
5
3816
conflict
saved but colored
2
1
4
5
3816
11
12
13
9
15
7
conflict
saved and uncolored
2
1
4
5
3816
11
12
13
9
15
10
67
colored by node 10
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
only node uncolored
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
18
choose farthest component
sorted list nearest neighbors2 1,4,53 1,8,161 2,3,168 3,11,125 1,2,411 8,12,134 2,5,716 1,3,512 8,9,1115 1,10,1110 2,6,1514 3,16,1717 5,14,1613 11,12,157 2,4,66 2,7,109 8,12,1418 4,5,17
16 / 27 Gwendal Simon Network Locality in Distributed Services
Approximate Algorithm
1
2
3
4
5
67
8
9
10
11
12
13
14
15
16
17
18
2
1
4
5
optimized2
1
4
5
3816
optimized
2
1
4
5
3816
conflict
saved but colored
2
1
4
5
3816
11
12
13
9
15
7
conflict
saved and uncolored
2
1
4
5
3816
11
12
13
9
15
10
67
colored by node 10
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
only node uncolored
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
18
choose farthest component
sorted list nearest neighbors2 1,4,53 1,8,161 2,3,168 3,11,125 1,2,411 8,12,134 2,5,716 1,3,512 8,9,1115 1,10,1110 2,6,1514 3,16,1717 5,14,1613 11,12,157 2,4,66 2,7,109 8,12,1418 4,5,17
16 / 27 Gwendal Simon Network Locality in Distributed Services
Approximate Algorithm
1
2
3
4
5
67
8
9
10
11
12
13
14
15
16
17
18
2
1
4
5
optimized2
1
4
5
3816
optimized
2
1
4
5
3816
conflict
saved but colored
2
1
4
5
3816
11
12
13
9
15
7
conflict
saved and uncolored
2
1
4
5
3816
11
12
13
9
15
10
67
colored by node 10
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
only node uncolored
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
18
choose farthest component
sorted list nearest neighbors2 1,4,53 1,8,161 2,3,168 3,11,125 1,2,411 8,12,134 2,5,716 1,3,512 8,9,1115 1,10,1110 2,6,1514 3,16,1717 5,14,1613 11,12,157 2,4,66 2,7,109 8,12,1418 4,5,17
16 / 27 Gwendal Simon Network Locality in Distributed Services
Approximate Algorithm
1
2
3
4
5
67
8
9
10
11
12
13
14
15
16
17
18
2
1
4
5
optimized2
1
4
5
3816
optimized
2
1
4
5
3816
conflict
saved but colored
2
1
4
5
3816
11
12
13
9
15
7
conflict
saved and uncolored
2
1
4
5
3816
11
12
13
9
15
10
67
colored by node 10
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
only node uncolored
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
18
choose farthest component
sorted list nearest neighbors2 1,4,53 1,8,161 2,3,168 3,11,125 1,2,411 8,12,134 2,5,716 1,3,512 8,9,1115 1,10,1110 2,6,1514 3,16,1717 5,14,1613 11,12,157 2,4,66 2,7,109 8,12,1418 4,5,17
16 / 27 Gwendal Simon Network Locality in Distributed Services
Approximate Algorithm
1
2
3
4
5
67
8
9
10
11
12
13
14
15
16
17
18
2
1
4
5
optimized2
1
4
5
3816
optimized
2
1
4
5
3816
conflict
saved but colored
2
1
4
5
3816
11
12
13
9
15
7
conflict
saved and uncolored
2
1
4
5
3816
11
12
13
9
15
10
67
colored by node 10
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
only node uncolored
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
18
choose farthest component
sorted list nearest neighbors2 1,4,53 1,8,161 2,3,168 3,11,125 1,2,411 8,12,134 2,5,716 1,3,512 8,9,1115 1,10,1110 2,6,1514 3,16,1717 5,14,1613 11,12,157 2,4,66 2,7,109 8,12,1418 4,5,17
16 / 27 Gwendal Simon Network Locality in Distributed Services
Approximate Algorithm
1
2
3
4
5
67
8
9
10
11
12
13
14
15
16
17
18
2
1
4
5
optimized2
1
4
5
3816
optimized
2
1
4
5
3816
conflict
saved but colored
2
1
4
5
3816
11
12
13
9
15
7
conflict
saved and uncolored
2
1
4
5
3816
11
12
13
9
15
10
67
colored by node 10
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
only node uncolored
2
1
4
5
3816
11
12
13
9
15
10
6
1417
7
18
choose farthest component
sorted list nearest neighbors2 1,4,53 1,8,161 2,3,168 3,11,125 1,2,411 8,12,134 2,5,716 1,3,512 8,9,1115 1,10,1110 2,6,1514 3,16,1717 5,14,1613 11,12,157 2,4,66 2,7,109 8,12,1418 4,5,17
16 / 27 Gwendal Simon Network Locality in Distributed Services
Proof
2
1
4
5
38 16
11
12
13
9
15
10
6
1417
7
18
2 4
7
i
4
i′18
i
14
i′
17
j1
d̄(i), cost to i ’s k − 1 nearest neighbors.d(i), rainbow cost of i .
optimized node: d(i) = d̄(i)
node conflicting with its nearest neighbor:
optimized node at one hop
d(i) ≤ (k − 2)d̄(i)
node with two conflicting nearest neighbors:
optimized node at two hops
d(i) ≤ ( 32 k − 5
2 )d̄(i)
17 / 27 Gwendal Simon Network Locality in Distributed Services
Proof
2
1
4
5
38 16
11
12
13
9
15
10
6
1417
7
18
2 4
7
i
4
i′18
i
14
i′
17
j1
d̄(i), cost to i ’s k − 1 nearest neighbors.d(i), rainbow cost of i .
optimized node: d(i) = d̄(i)
node conflicting with its nearest neighbor:
optimized node at one hop
d(i) ≤ (k − 2)d̄(i)
node with two conflicting nearest neighbors:
optimized node at two hops
d(i) ≤ ( 32 k − 5
2 )d̄(i)
17 / 27 Gwendal Simon Network Locality in Distributed Services
Proof
2
1
4
5
38 16
11
12
13
9
15
10
6
1417
7
18
2 4
7
i
4
i′
18
i
14
i′
17
j1
d̄(i), cost to i ’s k − 1 nearest neighbors.d(i), rainbow cost of i .
optimized node: d(i) = d̄(i)
node conflicting with its nearest neighbor:
optimized node at one hop
d(i) ≤ (k − 2)d̄(i)
node with two conflicting nearest neighbors:
optimized node at two hops
d(i) ≤ ( 32 k − 5
2 )d̄(i)
17 / 27 Gwendal Simon Network Locality in Distributed Services
Proof
2
1
4
5
38 16
11
12
13
9
15
10
6
1417
7
18
2 4
7
i
4
i′
18
i
14
i′
17
j1
d̄(i), cost to i ’s k − 1 nearest neighbors.d(i), rainbow cost of i .
optimized node: d(i) = d̄(i)
node conflicting with its nearest neighbor:
optimized node at one hop
d(i) ≤ (k − 2)d̄(i)
node with two conflicting nearest neighbors:
optimized node at two hops
d(i) ≤ ( 32 k − 5
2 )d̄(i)
17 / 27 Gwendal Simon Network Locality in Distributed Services
A Heuristic Algorithm
Idea: use the similarity with domatic partitiondomatic coloring of a proximity graph
1. build a k-nearest neighbor graph O(n · log n)
2. augment it into an interval graph O(n)
3. build the domatic partition O(n)
18 / 27 Gwendal Simon Network Locality in Distributed Services
A Heuristic Algorithm
Idea: use the similarity with domatic partitiondomatic coloring of a proximity graph
1. build a k-nearest neighbor graph O(n · log n)
2. augment it into an interval graph O(n)
3. build the domatic partition O(n)
18 / 27 Gwendal Simon Network Locality in Distributed Services
Another Heuristic Algorithm
Based on a k-nearest neighbor graph, two rounds1. explore surroundings: do not pick a component
hosted by a direct neighborhosted by a peer that considers you as a direct neighborhosted by one of its direct neighbors
2. try to maximize the benefitspick component satisfying in average the direct neighbors
19 / 27 Gwendal Simon Network Locality in Distributed Services
Another Heuristic Algorithm
Based on a k-nearest neighbor graph, two rounds1. explore surroundings: do not pick a component
hosted by a direct neighborhosted by a peer that considers you as a direct neighborhosted by one of its direct neighbors
2. try to maximize the benefitspick component satisfying in average the direct neighbors
19 / 27 Gwendal Simon Network Locality in Distributed Services
Simulations
20 / 27 Gwendal Simon Network Locality in Distributed Services
Configurations
Several contexts have been considered:network of latencies
select randomly n peers among 20, 000 entries⇒ minimize the global latency
network of Autonomous Systemsput µ peers into every ASinter-AS routing
⇒ minimize the cross-domain traffic
21 / 27 Gwendal Simon Network Locality in Distributed Services
Comparing to Exact Solutions
22 / 27 Gwendal Simon Network Locality in Distributed Services
Going Further
23 / 27 Gwendal Simon Network Locality in Distributed Services
Cross-Domain Gain
3,778
1,346
1,072 1,074 1,0853
4
5
6
Av
era
ge
Ho
ps
2,695 2,764 2,746
0
1
2
Random k-nearest Topo k-nearest Rela k-PUFLP
Av
era
ge
Ho
ps
Peering Transit
24 / 27 Gwendal Simon Network Locality in Distributed Services
Conclusion
25 / 27 Gwendal Simon Network Locality in Distributed Services
Only Preliminary Works
Many theoretical results can be obtained:relax assumptions (esp. capacity, number ofcomponents)study families of instancesbetter approximation
Many realistic variants can be formulated:take into account network architectureobjective of fairness
26 / 27 Gwendal Simon Network Locality in Distributed Services
Only Preliminary Works
Many theoretical results can be obtained:relax assumptions (esp. capacity, number ofcomponents)study families of instancesbetter approximation
Many realistic variants can be formulated:take into account network architectureobjective of fairness
26 / 27 Gwendal Simon Network Locality in Distributed Services
Any question?
<no image>
27 / 27 Gwendal Simon Network Locality in Distributed Services