15-744: computer networking l-7 software forwarding
TRANSCRIPT
![Page 1: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/1.jpg)
15-744: Computer Networking
L-7 Software Forwarding
![Page 2: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/2.jpg)
Software-Based Routers
• Motivation• Enabling innovation in networking research• Software data planes
• Readings:• OpenFlow: Enabling Innovation in Campus Networks• The Click Modular Router
• Optional reading• RouteBricks: Exploiting Parallelism To Scale Software
Routers
2
![Page 3: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/3.jpg)
Active Networking Recap
• Network API exposes capabilities • Processing, queues, storage
• Custom code/functions run on each packet
• E.g., conventional IP is best effort, dst based• When could this be insufficient?
3
![Page 4: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/4.jpg)
Two models of active networks
• “Capsule”• Packet carries code!
• Programmable router• Operator installs modules on router
• Pros/cons?
4
![Page 5: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/5.jpg)
Criticisms
• Too far removed from conventional networks• Upgrade/deployability?
• Capsule was considered insecure
• No killer apps (continues to be problem)
• Performance?
5
![Page 6: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/6.jpg)
Three logical stages (more hindsight)
• Active networking era• Case for “programmable” network devices
• “Separation” of control vs data era• Specifically about routing etc
• OpenFlow/Network OS era
6
![Page 7: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/7.jpg)
Network Management
Traffic Engineering PerformanceSecurity ComplianceResilience
7
![Page 8: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/8.jpg)
Problem: Toolbox is bad!
Traffic Engineering PerformanceSecurity ComplianceResilience
8
![Page 9: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/9.jpg)
Why: Toolbox is implicit in routers!
Traffic Engineering PerformanceSecurity ComplianceResilience
9
Motivation: Management is complex, expensive, fragileNeed: Direct control, expressive policy, network-wide views
![Page 10: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/10.jpg)
Solution
• Separate out the “data” and the “control”
• Open interface between control/data planes
• Logically centralized views• Simplifies optimization/policy management• Network-wide visibility
10
![Page 11: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/11.jpg)
Today: OpenFlow
Controller
Config Config
OpenFlow
![Page 12: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/12.jpg)
Next Lecture: ONIX
Controller
Config Config
E.g., ONIX, NOX, …
![Page 13: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/13.jpg)
OpenFlow: Motivation
• The Internet is a “success disaster”• Many successful applications• Critical for economy as a whole• Too huge a vested infrastructure• Vendors loathe to change anything
• Fear in community: “ossification” • New ideas cannot get deployed
![Page 14: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/14.jpg)
Driving questions
• Get our own operators comfortable with running network experiments
• Isolate experimental traffic from production traffic
• What is the functionality that enables innovation?
![Page 15: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/15.jpg)
Rejected alternatives
• Get vendors to support
• Use PC/Linux based network elements
• Existing research prototypes for programmable elements
![Page 16: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/16.jpg)
Their Path
• “Pragmatic compromise”
• Sacrifice generality for: • Performance• Cost• Vendor “buy-in”
![Page 17: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/17.jpg)
Three Basic Features in OpenFlow
Controller
Config Config
FlowTable
SecureChannel
OpenProtocol
![Page 18: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/18.jpg)
FlowTable Actions
• Forward on specific port/interface
• Forward to controller (encapsulated)
• Drop
• Forward legacy
• Future support: counters, modifiers
![Page 19: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/19.jpg)
What is nice
• Fits well with the TCAM abstraction
• Most vendors already have this
• They can just expose this without exposing internals
![Page 20: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/20.jpg)
Example Apps
• Ethane
• Amy’s own OSPF
• VLAN
• VoIP for Mobile
• Support for non-IP
![Page 21: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/21.jpg)
Driving questions: Did it achieve this?
• Get operators comfortable with running experimental?
• Isolate experimental traffic from production traffic?
• What is the functionality that can enable innovation?
![Page 22: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/22.jpg)
Software-Based Routers
• Enabling innovation in networking research• Software data planes
• Readings:• OpenFlow: Enabling Innovation in Campus Networks• The Click Modular Router
• Optional reading• RouteBricks: Exploiting Parallelism To Scale Software
Routers
22
![Page 23: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/23.jpg)
Click overview
• Modular architecture• Router = composition of modules• Router = data flow graph
• An element is the basic unit of processing
• Three key components of each element:• Ports• Configuration• Method interfaces
23
![Page 24: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/24.jpg)
Simple Tee Element
24
![Page 25: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/25.jpg)
Two types of “connections”
• Push• Source element has finished processing• Sends it downstream• E.g., FromDevice
• Pull• Destination is ready to process• Initiates packet transfer• E.g., ToDevice
25
![Page 26: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/26.jpg)
“Flow” of processing
26
![Page 27: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/27.jpg)
Click Config File
27
![Page 28: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/28.jpg)
Click Elements
28
![Page 29: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/29.jpg)
Other elements
• Packet Classification
• Scheduling
• Queueing
• Routing
• What you write…
29
![Page 30: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/30.jpg)
Idea: Polling
• Under heavy load, disable the network card’s interrupts
• Use polling instead• Ask if there is more work once you’ve done the first
batch
• Click paper we read – does pure polling
![Page 31: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/31.jpg)
Takeaways
• Click is a flexible modular router
• Shows that s/w x86 can get pretty good performance
• Extensible/modular
• Widely used in academia/research• Play with it!
31
![Page 32: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/32.jpg)
Software-Based Routers
• Enabling innovation in networking research• Software data planes
• Readings:• OpenFlow: Enabling Innovation in Campus Networks• The Click Modular Router
• Optional reading• RouteBricks: Exploiting Parallelism To Scale
Software Routers
32
![Page 33: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/33.jpg)
Building routers
• Fast
• Programmable
• custom statistics• filtering
• packet transformation
• …
33RouteBricks slides: Katerina Argyraki, 2009
![Page 34: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/34.jpg)
Why programmable routers
• New ISP services• intrusion detection, application
acceleration
• Simpler network monitoring • measure link latency, track down traffic
• New protocols• IP traceback, Trajectory Sampling, …
34
Enable flexible, extensible networks
![Page 35: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/35.jpg)
Today: fast or programmable
• Fast “hardware” routers• throughput : Tbps
• little programmability
• Programmable “software” routers• processing by general-purpose CPUs
• throughput < 10Gbps
35
![Page 36: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/36.jpg)
RouteBricks
• A router out of off-the-shelf PCs
• familiar programming environment
• large-volume manufacturing
• Can we build a Tbps router out of PCs?
36
![Page 37: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/37.jpg)
packet processing+
switching
Router =
• N: number of external router ports
• R: external line rate
R
R
R
R
R
R
R
R
37
N
![Page 38: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/38.jpg)
N
RR
A hardware router
• Processing at rate ~R per linecard
linecards linecards
38
![Page 39: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/39.jpg)
A hardware router
• Processing at rate ~R per linecard
• Switching at rate N x R by switch fabric
switch fabric
N
RR
linecards linecards
39
![Page 40: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/40.jpg)
commodity interconnect
RouteBricks
N
RR
• Processing at rate ~R per server
• Switching at rate ~R per server
servers servers
40
![Page 41: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/41.jpg)
Outline
• Interconnect
• Server optimizations
• Performance
41
![Page 42: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/42.jpg)
commodity interconnect
Requirements
N
RR
• Internal link rates < R• Per-server processing rate: c x R• Per-server fanout: constant
42
![Page 43: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/43.jpg)
A naive solution
N
R
RR
43
![Page 44: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/44.jpg)
A naive solution
N
RR R
44
• N external links of capacity R
• N2 internal links of capacity R
![Page 45: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/45.jpg)
Valiant load balancing (VLB)
N
R
R R/NR/N
45
![Page 46: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/46.jpg)
Valiant load balancing (VLB)
N
RR
46
• N external links of capacity R
• N2 internal links of capacity R
R/N R/N
2R/N
![Page 47: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/47.jpg)
Valiant load balancing (VLB)
N
RR R/N R/N
• Per-server processing rate: 3R
• W/ uniform traffic: 2R
47
![Page 48: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/48.jpg)
Per-server fanout?
N
R
48
![Page 49: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/49.jpg)
Per-server fanout?
N
R
• Increase server capacity
49
![Page 50: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/50.jpg)
Per-server fanout?
N
R
• Increase server capacity
50
![Page 51: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/51.jpg)
Per-server fanout?
N
R
• Increase server capacity
• Add intermediate nodes• k-degree n-stage butterfly
51
![Page 52: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/52.jpg)
Our solution: combination
• Assign max external ports per server
• Full mesh, if possible
• Extra servers, otherwise
52
![Page 53: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/53.jpg)
Valiant load balancing
+full mesh k-ary n-fly
Recap
N
RR
Per-server processing rate: 2R – 3R
53
![Page 54: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/54.jpg)
Outline
• Interconnect
• Server optimizations
• Performance
54
![Page 55: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/55.jpg)
Setup: NUMA architecture
I/O hub
Mem
Cores
Mem
• Nehalem architecture, QuickPath interconnect
• CPUs: 2 x [2.8GHz, 4 cores, 8MB L3 cache]
• NICs: 2 x Intel XFSR 2x10Gbps
• kernel-mode Click
Ports
55
![Page 56: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/56.jpg)
Single-server performance
I/O hub
Mem
Cores
Mem
Ports
56
• First try: 1.3 Gbps
![Page 57: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/57.jpg)
Problem #1: book-keeping
• Managing packet descriptors• moving between NIC and memory
• updating descriptor rings
• Solution: batch packet operations• NIC batches multiple packet descriptors
• CPU polls for multiple packets
57
![Page 58: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/58.jpg)
Single-server performance
I/O hub
Mem
Cores
Mem
Ports
58
• First try: 1.3 Gbps
• With batching: 3 Gbps
![Page 59: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/59.jpg)
Problem #2: queue access
CoresPorts
59
![Page 60: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/60.jpg)
Problem #2: queue access
60
• Rule #1: 1 core per port
![Page 61: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/61.jpg)
Problem #2: queue access
61
• Rule #1: 1 core per port
• Rule #2: 1 core per packet
![Page 62: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/62.jpg)
Problem #2: queue access
62
• Rule #1: 1 core per port
• Rule #2: 1 core per packet
![Page 63: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/63.jpg)
Problem #2: queue access
63
• Rule #1: 1 core per port
• Rule #2: 1 core per packet
![Page 64: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/64.jpg)
Problem #2: queue access
64
• Rule #1: 1 core per port
• Rule #2: 1 core per packet
queue
![Page 65: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/65.jpg)
Single-server performance
I/O hub
Mem
Cores
Mem
Ports
65
• First try: 1.3 Gbps
• With batching: 3 Gbps
• With multiple queues: 9.7 Gbps
![Page 66: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/66.jpg)
Recap
• State-of-the art hardware• NUMA architecture, multi-queue NICs
• Modified NIC driver• batching
• Careful queue-to-core allocation• one core per queue, per packet
66
![Page 67: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/67.jpg)
Outline
• Interconnect
• Server optimizations
• Performance
67
![Page 68: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/68.jpg)
Effect of application
68
• Throughput heavily depends on workload.
![Page 69: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/69.jpg)
Summary
• Vision of active networking
• Separating data plane and control plane
• Building software routers by starting with: • closed, commercial routers vs.
• commodity PCs
• Pros and cons?
69
![Page 70: 15-744: Computer Networking L-7 Software Forwarding](https://reader036.vdocuments.us/reader036/viewer/2022081516/56649d765503460f94a5796e/html5/thumbnails/70.jpg)
Next Lecture
• Software-Defined Networking• Readings:
• 4D: Read in full• Onix: Read intro• Ethane: Optional reading
70