berlin – november 10th, 2011 netfpga programmable networking for high-speed network prototypes,...
TRANSCRIPT
![Page 1: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/1.jpg)
Berlin – November 10th, 2011
NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and
Teaching
Presented by: Andrew W. Moore
(University of Cambridge)
CHANGE/OFELIABerlin, Germany
November 10th, 2011
http://NetFPGA.org
![Page 2: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/2.jpg)
Berlin – November 10th, 2011
Tutorial Outline
• Motivation– Introduction– The NetFPGA Platform
• Hardware Overview– NetFPGA 1G– NetFPGA 10G
• The Stanford Base Reference Router– Motivation: Basic IP review– Example 1: Reference Router running on the NetFPGA– Example 2: Understanding buffer size requirements using NetFPGA
• Community Contributions– Altera-DE4 NetFPGA Reference Router (UMassAmherst)– NetThreads (University of Toronto)
• Concluding Remarks
![Page 3: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/3.jpg)
Berlin – November 10th, 2011
Section I: Motivation
![Page 4: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/4.jpg)
Berlin – November 10th, 2011
NetFPGA = Networked FPGA
A line-rate, flexible, open networking platform for teaching and research
![Page 5: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/5.jpg)
Berlin – November 10th, 2011
NetFPGA 1G Board
NetFPGA consists of…
Four elements:
• NetFPGA board
• Tools + reference designs
• Contributed projects
• CommunityNetFPGA 10G Board
![Page 6: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/6.jpg)
Berlin – November 10th, 2011
NetFPGA 1G NetFPGA 10G
4 x 1Gbps Ethernet Ports
4 x 10Gbps SFP+
4.5 MB ZBT SRAM64 MB DDR2 SDRAM
27 MB QDRII-SRAM288 MB RLDRAM-II
PCI PCI Express x8
Virtex II-Pro 50 Virtex 5 TX240T
NetFPGA Board Comparison
![Page 7: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/7.jpg)
Berlin – November 10th, 2011
FPGA
Memory
1GE
1GE
1GE
1GE
NetFPGA board
PCI
CPU Memory
NetFPGA Board
PC with NetFPGA
NetworkingSoftwarerunning on a standard PC
A hardware acceleratorbuilt with Field Programmable Gate Arraydriving Gigabit network links
![Page 8: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/8.jpg)
Berlin – November 10th, 2011
Tools + Reference Designs
Tools:• Compile designs• Verify designs• Interact with hardware
Reference designs:• Router (HW)• Switch (HW)• Network Interface Card (HW)• Router Kit (SW)• SCONE (SW)
![Page 9: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/9.jpg)
Berlin – November 10th, 2011
Contributed Projects
More projects:http://netfpga.org/foswiki/NetFPGA/OneGig/ProjectTable
Project Contributor
OpenFlow switch Stanford University
Packet generator Stanford University
NetFlow Probe Brno University
NetThreads University of Toronto
zFilter (Sp)router Ericsson
Traffic Monitor University of Catania
DFA UMass Lowell
![Page 10: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/10.jpg)
Berlin – November 10th, 2011
Community
Wiki• Documentation
– User’s Guide– Developer’s Guide
• Encourage users to contribute
Forums• Support by users for users• Active community - 10s-100s of
posts/week
![Page 11: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/11.jpg)
Berlin – November 10th, 2011
International Community
Over 1,000 users, using 1,900 cards at150 universities in 32 countries
![Page 12: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/12.jpg)
Berlin – November 10th, 2011
NetFPGA’s Defining Characteristics
• Line-Rate– Processes back-to-back packets
• Without dropping packets • At full rate of Gigabit Ethernet Links
– Operating on packet headers • For switching, routing, and firewall rules
– And packet payloads• For content processing and intrusion prevention
• Open-source Hardware – Similar to open-source software
• Full source code available • BSD-Style License
– But harder, because • Hardware modules must meeting timing• Verilog & VHDL Components have more complex interfaces • Hardware designers need high confidence in specification of modules
![Page 13: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/13.jpg)
Berlin – November 10th, 2011
Test-Driven Design
• Regression tests– Have repeatable results – Define the supported features– Provide clear expectation on functionality
• Example: Internet Router– Drops packets with bad IP checksum– Performs Longest Prefix Matching on destination address– Forwards IPv4 packets of length 64-1500 bytes– Generates ICMP message for packets with TTL <= 1– Defines how packets with IP options or non IPv4
… and dozens more … Every feature is defined by a regression test
![Page 14: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/14.jpg)
Berlin – November 10th, 2011
Who, How, Why
Who uses the NetFPGA?– Teachers– Students– Researchers
How do they use the NetFPGA?– To run the Router Kit– To build modular reference designs
• IPv4 router• 4-port NIC• Ethernet switch, …
Why do they use the NetFPGA?– To measure performance of Internet systems– To prototype new networking systems
![Page 15: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/15.jpg)
Berlin – November 10th, 2011
Section II: Hardware Overview
![Page 16: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/16.jpg)
Berlin – November 10th, 2011
NetFPGA-1G
![Page 17: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/17.jpg)
Berlin – November 10th, 2011
Xilinx Virtex II Pro 50
• 53,000 Logic Cells• Block RAMs• Embedded PowerPC
![Page 18: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/18.jpg)
Berlin – November 10th, 2011
Network and Memory
• Gigabit Ethernet– 4 RJ45 Ports– Broadcom PHY
• Memories– 4.5MB Static RAM– 64MB DDR2 Dynamic
RAM
![Page 19: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/19.jpg)
Berlin – November 10th, 2011
Other IO
•PCI– Memory Mapped
Registers– DMA Packet Transferring
•SATA– Board to Board
communication
![Page 20: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/20.jpg)
Berlin – November 10th, 2011
NetFPGA-10G
• A major upgrade• State-of-the-art technology
![Page 21: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/21.jpg)
Berlin – November 10th, 2011
NetFPGA 1G NetFPGA 10G
4 x 1Gbps Ethernet Ports
4 x 10Gbps SFP+
4.5 MB ZBT SRAM64 MB DDR2 SDRAM
27 MB QDRII-SRAM288 MB RLDRAM-II
PCI PCI Express x8
Virtex II-Pro 50 Virtex 5 TX240T
Comparison
![Page 22: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/22.jpg)
Berlin – November 10th, 2011
10 Gigabit Ethernet
• 4 SFP+ Cages• AEL2005 PHY• 10G Support
– Direct Attach Copper– 10GBASE-R Optical
Fiber
• 1G Support– 1000BASE-T Copper– 1000BASE-X Optical
Fiber
![Page 23: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/23.jpg)
Berlin – November 10th, 2011
Others
• QDRII-SRAM– 27MB– Storing routing tables,
counters and statistics
• RLDRAM-II– 288MB– Packet Buffering
• PCI Express x8– PC Interface
• Expansion Slot
![Page 24: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/24.jpg)
Berlin – November 10th, 2011
Xilinx Virtex 5 TX240T
• Optimized for ultra high-bandwidth applications
• 48 GTX Transceivers
• 4 hard Tri-mode Ethernet MACs
• 1 hard PCI Express Endpoint
![Page 25: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/25.jpg)
Berlin – November 10th, 2011
Beyond Hardware
• NetFPGA-10G Board• Xilinx EDK based IDE• Reference designs with
ARM AXI4• Software (embedded
and PC)• Public Repository
(GitHub)• Public Wiki (PBWorks)
Reference Designs AXI4 IPs
Xilinx EDK
MicroBlaze SW PC SW
Wiki, GitHub, User Community
![Page 26: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/26.jpg)
Berlin – November 10th, 2011
NetFPGA-1G Cube Systems
• PCs assembled from parts– Stanford University– Cambridge University
• Pre-built systems available – Accent Technology Inc.
• Details are in the Guidehttp://netfpga.org/static/guide.html
![Page 27: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/27.jpg)
Berlin – November 10th, 2011
Rackmount NetFPGA-1G Servers
NetFPGA inserts in PCI or PCI-X slot
2U Server (Dell 2950)
Thanks: Brian Cashman for providing machine
1U Server (Accent Technology Inc.)
![Page 28: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/28.jpg)
Berlin – November 10th, 2011
Stanford NetFPGA-1G Cluster
Statistics• Rack of 40• 1U PCs with
NetFPGAs
• Managed • Power• Console• LANs
• Provides 4*40=160 Gbps of full line-rate processing bandwidth
![Page 29: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/29.jpg)
Berlin – November 10th, 2011
Section III: Network review
![Page 30: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/30.jpg)
Berlin – November 10th, 2011
Internet Protocol (IP)
Data
DataIP
Hdr
Eth Hdr
DataIP
Hdr
Data to betransmitted:
IP packets:
EthernetFrames:
DataIP
HdrData
IP Hdr
Eth Hdr
DataIP
HdrEth Hdr
DataIP
Hdr
…
…
![Page 31: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/31.jpg)
Berlin – November 10th, 2011
Internet Protocol (IP)
Data
DataIP
Hdr…
16 3241
Options (if any)
Destination Address
Source Address
Header ChecksumProtocolTTL
Fragment OffsetFlagsFragment ID
Total Packet LengthT.ServiceHLenVer
20 b
ytes
![Page 32: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/32.jpg)
Berlin – November 10th, 2011
Basic operation of an IP routerR3
A
B
C
R1
R2
R4 D
E
FR5
R5F
R3E
R3D
Next HopDestination
D
![Page 33: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/33.jpg)
Berlin – November 10th, 2011
Basic operation of an IP router
A
B
C
R1
R2
R3
R4 D
E
FR5
![Page 34: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/34.jpg)
Berlin – November 10th, 2011
Forwarding tables
Entry Destination Port
12⋮
232
0.0.0.00.0.0.1
⋮255.255.255.255
12⋮12
~ 4 billion entries
Naïve approach:One entry per address
Improved approach:Group entries to reduce table size
Entry Destination Port
12⋮50
0.0.0.0 – 127.255.255.255128.0.0.1 – 128.255.255.255
⋮248.0.0.0 – 255.255.255.255
12⋮12
IP address 32 bits wide → ~ 4 billion unique address
![Page 35: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/35.jpg)
Berlin – November 10th, 2011
IP addresses as a line
0 232-1
Entry Destination Port
12345
StanfordBerkeley
North AmericaAsia
Everywhere (default)
12345
All IP addresses
North AmericaAsia
BerkeleyStanford
Your computer My computer
![Page 36: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/36.jpg)
Berlin – November 10th, 2011
Longest Prefix Match (LPM)
Entry Destination Port
12345
StanfordBerkeley
North AmericaAsia
Everywhere (default)
12345
Universities
Continents
Planet
DataTo:
Stanford
Matching entries:•Stanford•North America•Everywhere
Most specific
![Page 37: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/37.jpg)
Berlin – November 10th, 2011
Longest Prefix Match (LPM)
Entry Destination Port
12345
StanfordBerkeley
North AmericaAsia
Everywhere (default)
12345
Universities
Continents
Planet
DataTo:
Canada
Matching entries:•North America•Everywhere
Most specific
![Page 38: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/38.jpg)
Berlin – November 10th, 2011
Implementing Longest Prefix Match
Entry Destination Port
12345
StanfordBerkeley
North AmericaAsia
Everywhere (default)
12345
Most specific
Least specific
Searching
FOUND
![Page 39: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/39.jpg)
Berlin – November 10th, 2011
Basic components of an IP router
Control Plane
Data Planeper-packet processing
SwitchingForwarding
Table
Routing Table
Routing Protocols
Management& CLI
Softw
areH
ardware
Queuing
![Page 40: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/40.jpg)
Berlin – November 10th, 2011
IP router components in NetFPGA
SCONE
Routing Table
Routing Protocols
Management& CLI
Output PortLookup
ForwardingTable
InputArbiter
OutputQueues
Switching Queuing
Linux
Routing Table
Routing Protocols
Management& CLI
Router Kit
OR
Softw
areH
ardware
![Page 41: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/41.jpg)
Berlin – November 10th, 2011
Section IV: Example I
![Page 42: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/42.jpg)
Berlin – November 10th, 2011
Operational IPv4 router
Control Plane
Data Planeper-packet processing
Softw
areH
ardware
Routing Table
Routing Protocols
Management& CLI
SCONE
SwitchingForwarding
TableQueuing
Reference router
Java GUI
![Page 43: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/43.jpg)
Berlin – November 10th, 2011
Streaming video
![Page 44: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/44.jpg)
Berlin – November 10th, 2011
Streaming video
PC & NetFPGA(NetFPGA in PC)
NetFPGA runningreference router
![Page 45: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/45.jpg)
Berlin – November 10th, 2011
Streaming video
Video streaming over shortest path
Videoclient
Videoserver
![Page 46: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/46.jpg)
Berlin – November 10th, 2011
Streaming video
Videoclient
Videoserver
![Page 47: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/47.jpg)
Berlin – November 10th, 2011
Observing the routing tables
Columns:•Subnet address•Subnet mask•Next hop IP•Output ports
![Page 48: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/48.jpg)
Berlin – November 10th, 2011
Example 1
http://www.youtube.com/watch?v=xU5DM5Hzqes
![Page 49: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/49.jpg)
Berlin – November 10th, 2011
Review Exercise 1
NetFPGA as IPv4 router:•Reference hardware + SCONE software•Routing protocol discovers topology
Example 1:•Ring topology•Traffic flows over shortest path•Broken link: automatically route around failure
![Page 50: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/50.jpg)
Berlin – November 10th, 2011
Section IV: Example II
![Page 51: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/51.jpg)
Berlin – November 10th, 2011
Buffers in Routers
• Internal Contention
• Congestion
• Pipelining
![Page 52: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/52.jpg)
Berlin – November 10th, 2011
Buffers in Routers
Rx
Rx
Rx
Tx
Tx
Tx
![Page 53: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/53.jpg)
Berlin – November 10th, 2011
Buffers in Routers
• So how large should the buffers be?
Buffer size matters– End-to-end delay
• Transmission, propagation, and queueing delay• The only variable part is queueing delay
– Router architecture• Board space, power consumption, and cost• On chip buffers: higher density, higher capacity• Optical buffers: all-optical routers
![Page 54: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/54.jpg)
Berlin – November 10th, 2011
Buffer Sizing Story
![Page 55: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/55.jpg)
Rule for adjusting W– If an ACK is received: W ← W+1/W– If a packet is lost: W ← W/2
Why 2TxC for a single TCP Flow?
Only W packets may be outstanding
![Page 56: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/56.jpg)
Berlin – November 10th, 2011
![Page 57: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/57.jpg)
Berlin – November 10th, 2011
Rule-of-thumb – Intuition
Rule for adjusting W If an ACK is received: W ← W+1/W If a packet is lost: W ← W/2
Only W packets may be outstanding
Source Dest
t
Window size
![Page 58: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/58.jpg)
Berlin – November 10th, 2011
Synchronized Flows Many TCP Flows• Aggregate window has
same dynamics• Therefore buffer occupancy
has same dynamics• Rule-of-thumb still holds.
• Independent, desynchronized
• Central limit theorem says the aggregate becomes Gaussian
• Variance (buffer size) decreases as N increases
Small Buffers – Intuition
ProbabilityDistribution
t
Buffer Size
t
![Page 59: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/59.jpg)
Berlin – November 10th, 2011
Poisson Traffic Smooth Traffic• Theory. For Poisson arrivals
tiny buffers are enough.
• Example: ρ = 80%, B = 20 pkts loss < 1%
• Loss independent of link rate, RTT, number of flows, etc.
• Question. Can we make traffic look like Poisson when it arrives to the core routers?
• Assumptions: – Minimum distance between
consecutive packets of the same flow;
– Desynchronized flows– Random and independent start
times for flows• Under these assumptions
traffic is be smooth-enough.
• In practice:– Slow access links– TCP Pacing
Tiny Buffers – Intuition
M/D/1Poisson
B
D loss < ρB
![Page 60: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/60.jpg)
Berlin – November 10th, 2011
Buffer Sizing Experiments are Difficult
Problem• Convincing network operators not easy• Packet drops are scary• Varying traffic (shape, load, ...)
extremely difficult• Tiny buffers: no guarantees on
assumptions – i.e. slow access or pacing
![Page 61: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/61.jpg)
Berlin – November 10th, 2011
Using NetFPGA to explore buffer size
• Need to reduce buffer size and measure occupancy
• Alas, not possible in commercial routers• So, we will use the NetFPGA instead
Objective:– Use the NetFPGA to understand how large a
buffer we need for a single TCP flow.
![Page 62: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/62.jpg)
Berlin – November 10th, 2011
Reference Router Pipeline
• Five stages– Input interfaces– Input arbitration– Routing decision and
packet modification– Output queuing– Output interfaces
• Packet-based module interface
• Pluggable design
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter
Output Port Lookup
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
![Page 63: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/63.jpg)
Berlin – November 10th, 2011
Extending the Reference Pipeline
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter
Output Port Lookup
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
RateLimiter
![Page 64: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/64.jpg)
Berlin – November 10th, 2011
Extending the Reference Pipeline
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter
Output Port Lookup
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
RateLimiter
Event Capture
![Page 65: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/65.jpg)
Berlin – November 10th, 2011
Enhanced Router Pipeline
Two modules added
1. Event Capture to capture output queue events (writes, reads, drops)
2. Rate Limiter to create a bottleneck
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
MACRxQ
CPURxQ
Input Arbiter
Output Port Lookup
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
MACTxQ
CPUTxQ
Output Queues
RateLimiter
Event Capture
![Page 66: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/66.jpg)
Berlin – November 10th, 2011
Topology for Exercise 2
Iperf Client
IperfServer
Recall:
NetFPGA host PC is life-support: power & control
So:
The host PC may physically route its traffic through the local NetFPGA
PC & NetFPGA(NetFPGA in PC)
NetFPGA runningextended reference router
nf2c2
eth1
nf2c1
eth2
![Page 68: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/68.jpg)
Berlin – November 10th, 2011
Review
NetFPGA as flexible platform:•Reference hardware + SCONE software•new modules: event capture and rate-limiting
Example 2:Client Router Server topology
– Observed router with new modules– Started tcp transfer, look at queue occupancy– Observed queue change in response to TCP ARQ
![Page 69: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/69.jpg)
Berlin – November 10th, 2011
Section V: Community Contributions
![Page 70: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/70.jpg)
Berlin – November 10th, 2011
FPGA
Memory
1GE
1GE
1GE
1GE
Running the Router Kit
User-space development, 4x1GE line-rate forwarding
PCI
CPU Memory
OSPF BGP
My Protocoluser
kernelRouting
Table
Usage #1
IPv4Router
1GE
1GE
1GE
1GE
FwdingTable
PacketBuffer
“Mirror”
![Page 71: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/71.jpg)
Berlin – November 10th, 2011
Altera-DE4 NetFPGA Reference Router
• Migration of NetFPGA infrastructure to DE4 Stratix IV – 4X logic vs. Virtex 2
• PCI Express Gen2 – 5.0Gbps/lane data • External DDR2 RAM – 8-Gbyte capacity.• Status: Functional – basic router
performance matches current NetFPGA• Lots of logic for additional functions• Russ Tessier ([email protected])
http://keb302.ecs.umass.edu/de4web/DE4_NetFPGA/
UMassAmherst
This provides a template for all NetFPGA 1G projects
![Page 72: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/72.jpg)
Berlin – November 10th, 2011
FPGA
Memory
1GE
1GE
1GE
1GE
Enhancing Modular Reference Designs
PCI
CPU Memory
Usage #2
NetFPGA Driver
Java GUIFront Panel(Extensible)
PW-OSPF
In QMgmt
IPLookup
L2Parse
L3Parse
Out QMgmt
1GE
1GE
1GE
1GEVerilog modules interconnected by FIFO interfaces
MyBlock
VerilogEDA Tools
(Xilinx, Mentor, etc.)
1. Design2. Simulate3. Synthesize4. Download
![Page 73: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/73.jpg)
Berlin – November 10th, 2011
FPGA
Memory
1GE
1GE
1GE
1GE
Creating new systems
PCI
CPU Memory
Usage #3
NetFPGA Driver
1GE
1GE
1GE
1GE
My Design
(1GE MAC is soft/replaceable)
VerilogEDA Tools
(Xilinx, Mentor, etc.)
1. Design2. Simulate3. Synthesize4. Download
![Page 74: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/74.jpg)
Berlin – November 10th, 2011 74
NetThreads, NetThreads-RE, NetTM
Martin LabrecqueGregory Steffan
ECE Dept.
Geoff SalmonMonia GhobadiYashar Ganjali
CS Dept.U. of Toronto
•Efficient multithreaded design–Parallel threads deliver performance
•System Features–System is easy to program in C–Time to results is very short
![Page 75: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/75.jpg)
Berlin – November 10th, 2011 75
FPGA
Soft processors: processors in the FPGA fabricUser uploads program to soft processorEasier to program software than hardware in the FPGACould be customized at the instruction level
Processor(s)DDR controller
Ethernet MAC
Soft Processors in FPGAs
![Page 76: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/76.jpg)
Berlin – November 10th, 2011
NetThreads
NetThreads, NetThreads-RE & NetTM available with supporting software at:
http://www.netfpga.org/foswiki/bin/view/NetFPGA/OneGig/NetThreadshttp://www.netfpga.org/foswiki/bin/view/NetFPGA/OneGig/NetThreads
REhttp://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/NetTM
Martin [email protected]
![Page 77: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/77.jpg)
Berlin – November 10th, 2011
Section VI: What to do next?
![Page 78: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/78.jpg)
Berlin – November 10th, 2011
To get started with your project
1. Get familiar with hardware description language
2. Prepare for your project
b) Get a hands-on tutorial
a) Learn NetFPGA by yourself
![Page 79: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/79.jpg)
Berlin – November 10th, 2011
Learn by yourself
Users Guide
NetFPGA website (www.netfpga.org)
![Page 80: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/80.jpg)
Berlin – November 10th, 2011
Learn by yourself
Developers Guide
NetFPGA website (www.netfpga.org)
Forums
![Page 81: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/81.jpg)
Berlin – November 10th, 2011
Support for NetFPGA enhancements provided by
Learn by Yourself
Online tutor – coming soon!
![Page 82: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/82.jpg)
Berlin – November 10th, 2011
Get a hands-on tutorial
Stanford
Cambridge
![Page 83: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/83.jpg)
Berlin – November 10th, 2011
Get a hands-on tutorial
Events
NetFPGA website (www.netfpga.org)
![Page 84: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/84.jpg)
Berlin – November 10th, 2011
Section VII: Conclusion
![Page 85: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/85.jpg)
Berlin – November 10th, 2011
Conclusions
• NetFPGA Provides– Open-source, hardware-accelerated Packet Processing– Modular interfaces arranged in reference pipeline – Extensible platform for packet processing
• NetFPGA Reference Code Provides– Large library of core packet processing functions– Scripts and GUIs for simulation and system operation– Set of Projects for download from repository
• The NetFPGA Base Code– Well defined functionality defined by regression tests– Function of the projects documented in the Wiki Guide
![Page 86: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/86.jpg)
Berlin – November 10th, 2011
Nick McKeown, Glen Gibb, Jad Naous, David Erickson, G. Adam Covington, John W. Lockwood, Jianying Luo, Brandon Heller, Paul
Hartke, Neda Beheshti, Sara Bolouki, James Zeng, Jonathan Ellithorpe, Sachidanandan Sambandan, Eric Lo
Acknowledgments
NetFPGA Team at Stanford University (Past and Present):
NetFPGA Team at University of Cambridge (Past and Present):
Andrew Moore, Shahbaz Muhammad, David Miller, Martin Zadnik
All Community members (including but not limited to):
Paul Rodman, Kumar Sanghvi, Wojciech A. Koszek, Yahsar Ganjali, Martin Labrecque, Jeff Shafer,
Eric Keller , Tatsuya Yabe, Bilal Anwer,Yashar Ganjali, Martin Labrecque
Kees Vissers, Michaela Blott, Shep Siegel
![Page 87: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/87.jpg)
Berlin – November 10th, 2011
Special thanks to our Partners:
Other NetFPGA Tutorial Presented At:
SIGMETRICS
Ram Subramanian, Patrick Lysaght, Veena Kumar, Paul Hartke, Anna Acevedo
Xilinx University Program (XUP)
See: http://NetFPGA.org/tutorials/
![Page 88: Berlin – November 10th, 2011 NetFPGA Programmable Networking for High-Speed Network Prototypes, Research and Teaching Presented by: Andrew W. Moore (University](https://reader038.vdocuments.us/reader038/viewer/2022103121/56649c775503460f9492c1e2/html5/thumbnails/88.jpg)
Berlin – November 10th, 2011
Thanks to our Sponsors:• Support for the NetFPGA project has been
provided by the following companies and institutions
Disclaimer: Any opinions, findings, conclusions, or recommendations expressed in these materials do not necessarily reflect the views of the National Science Foundation or of any other sponsors supporting this project.