wresting control from bgp: scalable fine-grained route control
DESCRIPTION
Wresting Control from BGP: Scalable Fine-grained Route Control. Patrick Verkaik. Dan Pei, Tom Scholl, Aman Shaikh, Alex C. Snoeren, Kobus van der Merwe. UCSD / AT&T Research Usenix —June 22, 2007. What is route control?. Route control. Traffic. ?. destination. ISP. measurement. ?. - PowerPoint PPT PresentationTRANSCRIPT
Wresting Control from BGP: Scalable Fine-grained Route Control
UCSD / AT&T ResearchUsenix —June 22, 2007
Dan Pei, Tom Scholl, Aman Shaikh, Alex C. Snoeren,
Kobus van der Merwe
Patrick Verkaik
What is route control?
ISP
Routecontrol
?
?
Traffic
destination
Traffic
• ISPs need to control selection of alternate paths• … in response to dynamic network conditions
• ISPs need to control selection of alternate paths• … in response to dynamic network conditions
measurement
• Historically routing provides simple connectivity• But demands are changing: gaming, DDoS defense
• Historically routing provides simple connectivity• But demands are changing: gaming, DDoS defense
IRSCP: enabling route control
• IRSCP: Intelligent Route Service Control Point– Next step after RCP [Caesar et al.]
• Enable route control for inter-domain traffic:
– Automated, based on network conditions– Scalable to demands of large ISP– No changes to existing ISP infrastructure
Simple example: load balancing
Customer ISP
Traffic
Egress router R1 Ingress routerR5
• Border Gateway Protocol (BGP)• BGP defaults to hot potato routing• Effectively: shortest path routing
Overloaded link
Traffic on customer access links is balanced
R3
R2
R4
• Simple route control objective• Customers are asking for this• Yet BGP can’t do it!
• Simple route control objective• Customers are asking for this• Yet BGP can’t do it!
Customer 1
Why does BGP do that?
ISP 1
ISP 2
Customer 2
External BGP (eBGP) session
Internal BGP (iBGP) session
ISP router
External router
Select
routeSelec
t route
Select
route
Select
route
dest d
dest d
R1
R3
R2R4
• Hundreds of ISP routers making local decisions• Hot potato default policy:
– Ensures consistency– ..but precludes control over routing
• Hundreds of ISP routers making local decisions• Hot potato default policy:
– Ensures consistency– ..but precludes control over routing
IRSCP route control
• Abstraction: “for this destination, direct traffic from this ISP router to that ISP router”
• Automated, based on network conditions:route control application
• Backwards compatible, consistent and scalable
• Selective: allow default BGP decision where desired
Outline
• IRSCP architecture
• Routing consistency
• Implementation
• Evaluation
Customer 1
Route control abstraction
ISP 1
ISP 2
Customer 2Traffic
Egress links
RC App
iBGP session
RC App
IRSCP
Assign-ments
• RC App assigns egress links to ISP routers
• Speak BGP to routers• …using IRSCP
AssignRoute?
?
R4
R1
R3R2
Route failover
• What if a route for egress link fails?– RC application at relatively slow
timescale
– IRSCP must fail over instantly
• So application sends a list of egress links for each ISP router– We call this a ranking
Customer 1
Rankings
ISP 1
ISP 2
Customer 2
Traffic
RC App
IRSCP
Rankings
Rankings R1
R4
R3
R2
12
12
R2 R4
R3R1 Select route for
R1 and R3
Select route for
R2 and R4
• Rankings map traffic from ingress to egress arbitrarily
• And allow route fail-over at routing time-scale
• Rankings map traffic from ingress to egress arbitrarily
• And allow route fail-over at routing time-scale
What about scalability?
• IRSCP talks to many thousands of routers• Responsible for route decision for each ISP
router:– Computation– Single point of failure
• Maintaining BGP session for each router :– State per session– Each eBGP session may add a route
2-3 GB sufficient
Distributed IRSCP
Customer 1
Distributed IRSCP
ISP 1
ISP 2
Customer 2
• Multiple IRSCP servers:– To distribute BGP sessions– For geographic diversity
• Routers may peer with several IRSCP servers• IRSCP servers are replicas: exchange all routes
IRSCP
IRSCP
IRSCP
IRSCP session
R4
R1
R3R2
Outline
• IRSCP architecture
• Routing consistency
• Implementation
• Evaluation
Consistency
Forwarding anomalies:
Deflection
Traffic
R1
R2
R3
R2
Looping
R1
R3
Rankings must be “consistent”Rankings must be “consistent”
123
123
R1
R3
R2
R4
Example of inconsistent rankings
RC application checks consistency constraints on rankings
No anomalies
TrafficR1R2
Rankings
R3 R4Deflection
Outline
• IRSCP architecture
• Routing consistency
• Implementation
• Evaluation
Expl. rankeddecision process
Hot potato/BGP decision process
Routing information baseRoutes
Prototype implementation
IRSCPserver
RC App Rankings
R1
IRSCP server
R2
IRSCPserver
Import policy
Export policy
I1 I2
Outline
• IRSCP architecture• Routing consistency• Implementation• Evaluation
– Can IRSCP handle routing load in a real ISP?– Both explicitly ranked and hot potato
decision process
Methodology
• Emulation of realistic large ISP topology and routing load
• Connect IRSCP implementation to emulation of ISP
• Measure if our implementation handles the emulated load
Customer 1
ISP 2
Deployment Scenario
IRSCP240 external routers per POP
15 ISP routers per POP
40 POPs
POP
1 IRSCP server per POP
IRSCP session
BGP session
Throughput of IRSCP server depends on how many of each kind of session it has
Finding maximum throughput
IRSCP
Input update rate
Routeupdatereceiver
Mix of BGP and IRSCP sessionsbased on ISP scenario
IRSCP implementation:– 3.6-GHz Xeon– 4 GB memory
Search for maximum sustainable input rate:– Gradually increase input rate, sustaining for twenty minutes– Compute expected output rate given input rate– Once measured output rate falls behind, we’ve reached
maximum throughput
Output update rate
Routeupdate
generator
Multiplier
Maximum throughput
3600
Output rate
Input rateHot potato
Explicitly ranked2400
41000
27000
600
95% in real ISPAverage in real ISP
220updates/s
updates/s
• Flexibility of rankings has a cost• But IRSCP handles Tier-1 ISP routing load, and more
• Flexibility of rankings has a cost• But IRSCP handles Tier-1 ISP routing load, and more
Conclusion
• IRSCP route control platform:– Feedback of network conditions into route selection– Scalable, robust against failures, backward compatible– Powerful, yet safe ranking abstraction
• Enables new class of “route control application”:– Security– Traffic engineering– Customer-oriented services
Trials of IRSCP for several applications taking place in AT&T!
Alex
Aman
Questions?
Kobus
Tom
Dan
Patrick
BGP sessions
• As we saw, IRSCP speaks iBGP to ISP routers
• For full control, IRSCP must also speak eBGP to external routers
IRSCP
Multihop eBGPsession
iBGP session
I
E1
E2eBGPsession
Select
routeSelect
route for I
Example consistency constraint
If e1 outranks e2 at R1 then must also do so at router(e1 )=R3
• Two simple consistency constraints• Checked by RC application before sending to IRSCP• Example:
123
123
R1
R3
e1e2
e1
e2
R1R2
R3 R4e1 e2
R2
R4
ThroughputAchieved output rate
Estimated 95 perc. required input rate
Achieved input rate
Out of 255 routers per POP