programming the topology of networks - umd
TRANSCRIPT
![Page 1: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/1.jpg)
Programming the Topology of NetworksTechnology and Algorithms
Pierre Blanche Klaus-Tycho Foerster
Jeff Cox Jamie Gaudette
Nikhil Devanur Phillipa Gill
Mark Filer Madeleine Glick
Daniel Kilper Gireeja Ranade
Janardhan Kulkarni Houman Rastegarfar
Ratul Mahajan Stefan Schmid
Amar Phanishayee Rachee Singh
Manya Ghobadi
![Page 2: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/2.jpg)
The cloud infrastructure
Inefficiencies and waste in
the cloud infrastructure
Data centers Optical cables
2
![Page 3: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/3.jpg)
Network
topology
Traffic engineeringCapacity provisioning
Technology and algorithms to optimize network topology
3
![Page 4: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/4.jpg)
High-level idea
Networks with programmable topologies
A B
D C
1
A B
D C
1
1
11
Throughput: 2 units Throughput: 3 units
1
1
1
Dynamic topologyStatic topology
4
![Page 5: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/5.jpg)
• Challenging:
• Requires reconfigurable hardware technology
• Requires revisiting networking layer algorithms
• Impactful:
• Cheaper networks
• Higher throughput
Programmable topologies
5
![Page 6: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/6.jpg)
Talk outline
Data center networks
ProjecToR: Programming the
network topology [SIGCOMM’16]
Technology and algorithms to enable programmable
topologies in the cloud
Google data center
Wide-area networks
Programming the capacity of links[SIGCOMM’18, HotNets’17, OFC’16]
Level3 global backbone
6
![Page 7: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/7.jpg)
Today’s data center interconnects
A B C D
0 3 3 3
3 0 3 3
3 3 0 3
3 3 3 0
Ideal demand matrix:
uniform and static
Static capacity between
Top-of-Rack (ToR) pairs
0 6 6 0
0 0 0 0
0 0 0 0
0 0 0 0
Non-ideal demand matrix:
skewed and dynamic
A
B
C
D
A B C D A B C D
A
B
C
D10Gbps
10Gbps
0 0 0 0
0 0 0 0
8 0 6 0
0 0 0 0
0 0 0 0
0 0 0 7
0 0 0 7
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 12 8 0
0 6 6 0
0 0 0 0
0 0 0 0
0 0 0 0
7
![Page 8: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/8.jpg)
Need for a reconfigurable interconnect
Data:
• 200K servers across 4 production clusters
• Cluster sizes: 100 -- 2500 racks
Observation:
• Many rack pairs exchange little traffic
• Only some hot rack pairs are active
Implication:
• Static topology with uniform capacity:
• Over-provisioned for most rack pairs
• Under-provisioned for few others
Reconfigurable interconnect:
To dynamically provide additional capacity between hot rack pairs
8
![Page 9: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/9.jpg)
Our proposal: ProjecToR interconnect
9
• Free-space topology (programmable)
• Digital micromirror device to redirect light
• Disco-ball shaped mirror assembly to magnify reachability
Laser Photodetector
Static topology 9
![Page 10: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/10.jpg)
Digital Micromirror Device (DMD)
Array of micromirrors (10 um) Memory cell
10
![Page 11: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/11.jpg)
A 3-ToR ProjecToR interconnect prototype
ToR1ToR2
ToR3
Source laser
DMD
Mirrors
reflecting to
ToR2 and ToR3
ToR1ToR2
ToR3
11
![Page 12: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/12.jpg)
Routing algorithm
• We have a highly flexible topology allowing for millions of ways to connect lasers to photodetectors
• Ideal solution: fast changing topology to adapt to demand change
• Challenge: It takes 12μs to reprogram a link
ToR1 ToR1
ToR2
ToR3
ToR2
ToR3
lasers photodetectors
12
![Page 13: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/13.jpg)
Routing algorithm
• Two topology approach:
• Slow switching topology or dedicated topology
• Fast switching links or opportunistic links
ToR1
ToR2
ToR3
ToR1
ToR2
ToR3
ToR1 ToR1
ToR2
ToR3
ToR2
ToR3
ToR1
ToR2
ToR3
ToR1
ToR2
ToR3
lasers photodetectors
dedicated topology opportunistic links
13
![Page 14: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/14.jpg)
Routing packets
ToR1
ToR2
ToR3
ToR1
ToR2
ToR3
dedicated topology
2
3
3
3
Virtual output queues
K-shortest paths routing
opportunistic link
2
2
2
2
2
2
2
14
![Page 15: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/15.jpg)
Scheduling opportunistic links
• Given a set of potential links and current traffic demand, find a set of active opportunistic links
ToR1
ToR2
ToR3
ToR1
ToR2
ToR3
0 0 100
100 0 0
0 0 0
s
o
u
r
c
e
d e s t i n a t i o n
15
![Page 16: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/16.jpg)
Scheduling opportunistic links
• Standard switch scheduling problem
• Blossom matching
• BvN matrix decomposition
• Centralized scheduler
• Single tiered matching
0 100 0
0 0 100
100 0 0
s
o
u
r
c
e
d e s t i n a t i o n
input output
16
![Page 17: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/17.jpg)
• Standard switch scheduling problem
• Blossom matching
• BvN matrix decomposition
• Centralized scheduler
• Single tiered matching
Scheduling opportunistic links
input outputSrc ToRs Dst ToRs
Two-tiered
Decentralized
Extended the Gale-Shapely algorithm for finding stable matches [GS-1962]
Constant competitive against an offline optimal allocation17
![Page 18: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/18.jpg)
- Slow switching time
+ Reconfigurable
+ Switching time: 12μs
• Tail flow completion time
• Different traffic matrices
• Impact of switching time
• Impact of fan-out
Simulation results
0
5
10
15
20
25
30
35
40
20 30 40 50 60 70 80
Ave
rage
Flo
w C
om
ple
tio
n T
ime
(m
s)
Average Load (%)
- No reconfigurability
95%
ProjecToR
Fat tree [SIGCOMM’08]
FireFly
[SIGCOMM’14]
18
![Page 19: Programming the Topology of Networks - UMD](https://reader031.vdocuments.us/reader031/viewer/2022012510/618846d862b37a27b47e5c91/html5/thumbnails/19.jpg)
The key takeaway from this talk
Current assumption: Network topology is fixed
New world: Network topology is dynamicc3
c4
s
w
u v
y
tc7 c8
Problems to solve:
• Scheduling
• Capacity provisioning
• Traffic engineering
• Load-balancing
• Exciting: Unusual wealth of algorithms
• Challenging: Changes fundamental assumptions
• Impactful: Better efficiency ($/Gbps) 19